Commit Graph

107 Commits

Author SHA1 Message Date
eba63c27ef
Youtube source is now working (#5)
* added extra packages to help with parsing

* getting the core built for Youtube collection.  The feed can be pulled and starting to build the article object

* added some tests, reddit will need more love but youtube is starting off with more tests.  Starting to add Rod to pull missing values from the site

* Added rod to work with browser automation

* Moved the config inside the client as they can change within runtime and need to be refreshed on each client creation

* added more features to collect data per video and tests to support them.

* adding the cache and tests

* moved errors to var values at the top

* youtube is now getting collected and tests have been setup

* resolved an issue with the config

* setting up the first interface, its not used yet

* more updates to the cache service. Not finished yet and could see rework

* added logic to monitor Youtube.  Still basic logic that needs to be wired up to the database
2022-04-17 07:25:49 -07:00
e61870ec60
Create go-build.yml (#4) 2022-04-13 17:01:11 -07:00
4a947d89f2
removed unused import for build (#3) 2022-04-07 14:59:24 -07:00
2b959e140c
Features/reddit and dto (#2)
* updated db calls to new endpoints.  Not all are finished yet

* updated models to use the new dto objects to hide orm values

* updated reddit to build articles based on what type of post it is

* getting reddit ready to post the articles to the db

* Now able to post to the db under the new DTO object

* moved the reddit model to articles model to the struct and out of main
2022-04-07 14:53:40 -07:00
281fcb2d8a
got docker building with the correct binary (#1) 2022-04-02 12:09:43 -07:00
d8e85b1bd7 Getting the basics setup for the collector. It can pull reddit json and almost post the db api 2022-04-02 12:05:32 -07:00
c6620e3c88
Initial commit 2022-04-02 10:01:57 -07:00