Commit Graph

104 Commits

Author SHA1 Message Date
James Tombleson 9586c6a544 repositories now use context and have interfaces exposed 2024-04-28 11:39:25 -07:00
James Tombleson 7227744621 got the sources repo working 2024-04-28 10:02:57 -07:00
James Tombleson 15681d9d37 Almost done with DiscordWebHooks repo 2024-04-27 13:11:03 -07:00
James Tombleson 3d3b582e82 Articles can be created and working on pulling over the old queries 2024-04-27 07:44:41 -07:00
James Tombleson f6cc0a3d93 entity updated to reflect table 2024-04-27 07:44:20 -07:00
James Tombleson ba33d18525 Redefined what can be null and removed some values I am not sure matter anymore 2024-04-27 07:44:04 -07:00
jtom38 2b5a3cc8e4 Merge pull request 'features/added-user-tables' (#3) from features/added-user-tables into main
Reviewed-on: #3
2024-04-26 16:06:35 -07:00
James Tombleson 44b7d24e07 updated the rss source to use the new entity ref 2024-04-26 16:05:38 -07:00
James Tombleson 0948ef9fa2 the config will now just return a struct from the env because this wa made to run in a container 2024-04-26 16:05:17 -07:00
James Tombleson d35b955815 The handler class will now store the sql connection for now 2024-04-26 16:04:10 -07:00
James Tombleson 2fa47c957c pulled my user and refreshTokens tables from another project 2024-04-26 16:03:32 -07:00
James Tombleson d1a4d10df0 cleaned up the dto's and entities to make them more like what I am use to 2024-04-26 16:02:59 -07:00
James Tombleson 81699b6919 Reworked migrations and added the user and refresh token tables 2024-04-26 16:02:36 -07:00
James Tombleson 8ea3f5fef1 renamed the main go file to server 2024-04-26 16:02:14 -07:00
jtom38 7b6fad28a3 Merge pull request 'chi was replaced with echo' (#2) from features/moved-to-echo into main
Reviewed-on: #2
2024-04-23 22:18:40 -07:00
James Tombleson ab026ef471 chi was replaced with echo 2024-04-23 22:18:07 -07:00
jtom38 feb9895eb7 Merge pull request 'moving code into the internal package' (#1) from features/reorganize-project into master
Reviewed-on: #1
2024-04-23 07:16:08 -07:00
James Tombleson 543e8d8eab moving code into the internal package 2024-04-23 07:15:38 -07:00
James Tombleson b0790359d5
bySourceId now supports paging and defaults to newest items (#41) 2023-02-04 21:43:07 -08:00
James Tombleson 118b7eb5e2
updated ArticlesList to always list the newest by default (#40) 2023-02-04 20:15:10 -08:00
James Tombleson 0ae1de0d33
updated the biild to always run swag and sqlc before build: (#39) 2023-01-31 17:47:04 -08:00
James Tombleson 8704680e82
Features/route cleanup (#38)
* moved route context to the request and not passed down

* dtoClient now returns a pointer to help on memory

* moved funcs to use the dto pointer
2023-01-31 12:39:54 -08:00
James Tombleson 90e739a56e
Features/get route updates (#37)
* cleaned up get routes

* added a func to query articles by page

* removed unused imports

* merged page with listArticles as a optional
2023-01-31 08:19:23 -08:00
James Tombleson ada453e08a
Features/delete source and first dto (#36)
* updated db, added dto for ListSources, and added delete source

* updated from model > models

* updated to models

* sources now sends back a standard message

* updated subscription routes to have beter logid and swagger details

* moved the dto objects back to modles given they are not bound to the database

* cleaned up how we return the error

* cleaned up swag and updated models to take from the base apistatusmodel. less human errors this way

* cleaned up swag and updated models

* swag updated

* updated queue to return a router and also renamed it as it will hold all queue info later on

* removed config tag

* added subscription details route

* article routes have been moved to support dto

* updated discordwebhooks to use dto

* updated discordwebhookqueue to return details on the items via dto

* removed the example routes

* updated sources to use dto

* subscriptions moved to dto

* generated swag
2023-01-22 10:12:55 -08:00
James Tombleson a2147294d5
updated db, added dto for ListSources, and added delete source (#35) 2022-12-07 22:48:31 -08:00
James Tombleson ff4075383a
Features/subscription/features for portal (#34)
* added a route to delete subscriptions based on the ID given

* added a new route to find a record based on the name and source

* added a route to query Discord Web Hooks by Server and Channel names

* tested the endpoints and they seem good to test more

* updated some routes for subscriptions and formatted files

* removed debug file

* fixing some panic calls

* swag
2022-12-04 08:49:17 -08:00
James Tombleson c161658487
New endpoints for the portal to use (#31)
* added a route to delete subscriptions based on the ID given

* added a new route to find a record based on the name and source

* added a route to query Discord Web Hooks by Server and Channel names

* tested the endpoints and they seem good to test more
2022-11-30 21:43:53 -08:00
James Tombleson 94da578c82
replaced sql command to order the new articles by source (#30) 2022-11-06 19:07:44 -08:00
James Tombleson 337151b39d
updated articles route to order by pubdate and updated swagger (#29) 2022-11-06 17:17:46 -08:00
James Tombleson 63b19452e5
removing build from merge. Only needed for PR (#28) 2022-08-21 20:06:01 -07:00
James Tombleson ac8eeaea01
Added discord management that was missing (#27) 2022-08-21 20:02:45 -07:00
James Tombleson b082c904ea
updated swagger for the new twitch change (#26) 2022-08-21 11:24:08 -07:00
James Tombleson 3be2b52688
updated new twitch given it no longer needs a url param (#25) 2022-08-21 11:18:28 -07:00
James Tombleson 06e50f956a
removed custom tags on source creation. Generates errors that we dont need currently (#19) 2022-07-27 10:56:44 -07:00
James Tombleson 206251a3d1
added a new endpoint to get a list of sources by the source value (#18) 2022-07-26 13:27:34 -07:00
James Tombleson 66c74802d9
Find installed browser for Docker (#17)
* trying a different method to have rod find to brower on disk for docker

* updated rod to find the browser on disk.  This is for docker support
2022-07-14 14:27:40 -07:00
James Tombleson 9d662936d5
added chromium package to docker (#16) 2022-07-14 11:42:55 -07:00
James Tombleson 79f3383fdc
Feature/dynamic swagger uri (#15)
* made the swagger uri based on env

* updated swagger to reflect the dynamic uri

* updated extamples
2022-07-14 09:59:55 -07:00
James Tombleson f01a8ff592
Features/swagger update (#14)
* ArticlesByTag added

* format cleanup
2022-07-13 21:31:53 -07:00
James Tombleson 65f4281f92
Feature/deployment update (#13)
* updated to go 1.18.4 and added alpine dep.  Image tested locally

* added the first example of how to run the app

* added deployment notes

* updated bootup logic around the env and how to handle things

* swagger update
2022-07-13 15:52:25 -07:00
James Tombleson a1324ee1c1
Features/output discord (#12)
* basic output looks to be working

* cron was updated to add to the queue and post messages

* new route to make discord webhook subscriptions

* updated swag tags

* swag

* Updated delete subscription call

* removed the time value as it throws off the msg template

* updated logging

* updated swagger

* updated new subscription route

* Updated logging and remove items from the queue if they dont have a subscription

* updated getArticles to return the 50 newest for the portal

* added endpoint to see if an item exists already

* formatting

* updated listArticles

* added colors and updated the image

* Updated to use the pointer in twitch

* added the twitch login command to cron... it works now

* found a better way to disable http2 for reddit. Test worked right away too

* updated the cron tasks to run collected once and hour or longer depending on the service
2022-07-12 15:28:31 -07:00
James Tombleson 0e0058506a
Feature Flags (#11)
* added feature flags around background workers

* background workers are moved to a new package as outputs are starting to get added

* package name was updated

* updated refs to the new input package

* query and sql updates on routes

* moved the services and starting to add discord web hook

* query update
2022-06-30 14:54:58 -07:00
James Tombleson 713205bb03
Basic routes have been added (#10)
* basic routes are working with db context

* swagger is working along with swag gen

* cron was updated with a class and better db context, untested though

* sourcelist command added

* lost the pg package but added it back

* Updated the api startup for cron and api

* updated source routes and started to add article routes

* Updated cron add func calls

* updated swagger

* keeping articles basic for now as I dont need to pull them in yet

* swagger update

* added getarticlesbysourceid call

* adding the subscriptions table to track who to send notifications and where

* removed legacy columns from discordwebhooks that are no longer needed.

* added discord webhook routes

* updated routes

* Minor change to schema

* Updated routes to support subscriptions

* ignore .vscode
2022-06-19 22:02:44 -07:00
James Tombleson 77d4fdf01a
version bump (#9) 2022-06-14 10:26:05 -07:00
James Tombleson 75b66dd625
Feature/sql (#8)
* Using sqlc to generate queries and goose for migrations. The intial tests look good.

* moving the old calls away for now.  Might use this in a package later on.

* Added postgres driver

* Updated the dockerfile to support sql migrations

* added sqlc config file

* updated schema and starting a seed script

* updated models to use the database ones

* updated reddit cron to talk to the db

* added env for sql connection string

* got the reddit source working with the db and posting articles

* added sql packages

* added rule to ignore dev sql file

* added migration down statement for rolling back

* updated cron for reddit and youtube

* Updated reddit to follow a new standard pattern

* updated youtube to follow new patterns

* updated tests and brought them to the standard

* updated the seed migration

* all cron tasks should feed the db now

* updated app init

* bumped docker to 1.18.3

* disabled remote tests given secrets and lack of interfaces currently to run tests
2022-06-08 21:17:08 -07:00
James Tombleson 333a4f5345
Features/twitch (#7)
* ran go mod tity

* updated formatting and moved to using some common errors

* some common source errors

* bumped go version

* a bulk of the work has been completed to parse twitch videos/users

* Added twitch config values

* added common errors

* moved the scheduler to its own package to get it out of main and also start adding tests

* added a new err for invalid author images

* Updated an error used

* tests updated to avoid name duplication

* new func to replace the cached source recor and updated getcontent to return collected posts

* updated scheduler ref

* moved to services/cron
2022-05-15 21:48:23 -07:00
James Tombleson 11892b9a7b
Features/ffxiv (#6)
* starting the ffxiv reader

* working on getting the standard interface for sources based on the work for ffxiv

* got more of ffxiv working and updated tests

* Author and Description can be extracted and validated with tests

* added uuid package

* ffxiv core logic is working and testes updated to reflect it.

* Updated the scheduler with the current sources and moved them from main

* updated reddit to allow modern go to talk to the endpoint with a debug flag

* gave the func a better name

* cleaned up main

* Moved cache to its own package and updated tests"

* moved config to its own package and added basic tests

* updated imports

* minor update"

* interface update and cache model update

* updated the scheduler for basic services.  No DB calls yet

* updated db calls

* bypassed the reddit test as its flaky in github
2022-04-29 13:02:25 -07:00
James Tombleson eba63c27ef
Youtube source is now working (#5)
* added extra packages to help with parsing

* getting the core built for Youtube collection.  The feed can be pulled and starting to build the article object

* added some tests, reddit will need more love but youtube is starting off with more tests.  Starting to add Rod to pull missing values from the site

* Added rod to work with browser automation

* Moved the config inside the client as they can change within runtime and need to be refreshed on each client creation

* added more features to collect data per video and tests to support them.

* adding the cache and tests

* moved errors to var values at the top

* youtube is now getting collected and tests have been setup

* resolved an issue with the config

* setting up the first interface, its not used yet

* more updates to the cache service. Not finished yet and could see rework

* added logic to monitor Youtube.  Still basic logic that needs to be wired up to the database
2022-04-17 07:25:49 -07:00
James Tombleson e61870ec60
Create go-build.yml (#4) 2022-04-13 17:01:11 -07:00
James Tombleson 4a947d89f2
removed unused import for build (#3) 2022-04-07 14:59:24 -07:00