James Tombleson
eba63c27ef
* added extra packages to help with parsing * getting the core built for Youtube collection. The feed can be pulled and starting to build the article object * added some tests, reddit will need more love but youtube is starting off with more tests. Starting to add Rod to pull missing values from the site * Added rod to work with browser automation * Moved the config inside the client as they can change within runtime and need to be refreshed on each client creation * added more features to collect data per video and tests to support them. * adding the cache and tests * moved errors to var values at the top * youtube is now getting collected and tests have been setup * resolved an issue with the config * setting up the first interface, its not used yet * more updates to the cache service. Not finished yet and could see rework * added logic to monitor Youtube. Still basic logic that needs to be wired up to the database
13 lines
342 B
Modula-2
13 lines
342 B
Modula-2
module github.com/jtom38/newsbot/collector
|
|
|
|
go 1.16
|
|
|
|
require (
|
|
github.com/PuerkitoBio/goquery v1.8.0 // indirect
|
|
github.com/go-chi/chi/v5 v5.0.7 // indirect
|
|
github.com/go-rod/rod v0.105.1 // indirect
|
|
github.com/joho/godotenv v1.4.0 // indirect
|
|
github.com/mmcdole/gofeed v1.1.3 // indirect
|
|
github.com/robfig/cron/v3 v3.0.1 // indirect
|
|
)
|