jtom38/newsbot-api
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11892b9a7b
newsbot-api/go.mod

14 lines
385 B
Modula-2
Raw Normal View History

Getting the basics setup for the collector. It can pull reddit json and almost post the db api
2022-04-02 12:05:32 -07:00
module github.com/jtom38/newsbot/collector
go 1.16
require (
Youtube source is now working (#5) * added extra packages to help with parsing * getting the core built for Youtube collection. The feed can be pulled and starting to build the article object * added some tests, reddit will need more love but youtube is starting off with more tests. Starting to add Rod to pull missing values from the site * Added rod to work with browser automation * Moved the config inside the client as they can change within runtime and need to be refreshed on each client creation * added more features to collect data per video and tests to support them. * adding the cache and tests * moved errors to var values at the top * youtube is now getting collected and tests have been setup * resolved an issue with the config * setting up the first interface, its not used yet * more updates to the cache service. Not finished yet and could see rework * added logic to monitor Youtube. Still basic logic that needs to be wired up to the database
2022-04-17 07:25:49 -07:00
github.com/PuerkitoBio/goquery v1.8.0 // indirect
Getting the basics setup for the collector. It can pull reddit json and almost post the db api
2022-04-02 12:05:32 -07:00
github.com/go-chi/chi/v5 v5.0.7 // indirect
Youtube source is now working (#5) * added extra packages to help with parsing * getting the core built for Youtube collection. The feed can be pulled and starting to build the article object * added some tests, reddit will need more love but youtube is starting off with more tests. Starting to add Rod to pull missing values from the site * Added rod to work with browser automation * Moved the config inside the client as they can change within runtime and need to be refreshed on each client creation * added more features to collect data per video and tests to support them. * adding the cache and tests * moved errors to var values at the top * youtube is now getting collected and tests have been setup * resolved an issue with the config * setting up the first interface, its not used yet * more updates to the cache service. Not finished yet and could see rework * added logic to monitor Youtube. Still basic logic that needs to be wired up to the database
2022-04-17 07:25:49 -07:00
github.com/go-rod/rod v0.105.1 // indirect
Features/ffxiv (#6) * starting the ffxiv reader * working on getting the standard interface for sources based on the work for ffxiv * got more of ffxiv working and updated tests * Author and Description can be extracted and validated with tests * added uuid package * ffxiv core logic is working and testes updated to reflect it. * Updated the scheduler with the current sources and moved them from main * updated reddit to allow modern go to talk to the endpoint with a debug flag * gave the func a better name * cleaned up main * Moved cache to its own package and updated tests" * moved config to its own package and added basic tests * updated imports * minor update" * interface update and cache model update * updated the scheduler for basic services. No DB calls yet * updated db calls * bypassed the reddit test as its flaky in github
2022-04-29 13:02:25 -07:00
github.com/google/uuid v1.3.0 // indirect
Getting the basics setup for the collector. It can pull reddit json and almost post the db api
2022-04-02 12:05:32 -07:00
github.com/joho/godotenv v1.4.0 // indirect
Youtube source is now working (#5) * added extra packages to help with parsing * getting the core built for Youtube collection. The feed can be pulled and starting to build the article object * added some tests, reddit will need more love but youtube is starting off with more tests. Starting to add Rod to pull missing values from the site * Added rod to work with browser automation * Moved the config inside the client as they can change within runtime and need to be refreshed on each client creation * added more features to collect data per video and tests to support them. * adding the cache and tests * moved errors to var values at the top * youtube is now getting collected and tests have been setup * resolved an issue with the config * setting up the first interface, its not used yet * more updates to the cache service. Not finished yet and could see rework * added logic to monitor Youtube. Still basic logic that needs to be wired up to the database
2022-04-17 07:25:49 -07:00
github.com/mmcdole/gofeed v1.1.3 // indirect
Getting the basics setup for the collector. It can pull reddit json and almost post the db api
2022-04-02 12:05:32 -07:00
github.com/robfig/cron/v3 v3.0.1 // indirect
)
Reference in New Issue Copy Permalink