This is the collection service of newsbot to pull articles from the web.
James Tombleson
eba63c27ef
* added extra packages to help with parsing * getting the core built for Youtube collection. The feed can be pulled and starting to build the article object * added some tests, reddit will need more love but youtube is starting off with more tests. Starting to add Rod to pull missing values from the site * Added rod to work with browser automation * Moved the config inside the client as they can change within runtime and need to be refreshed on each client creation * added more features to collect data per video and tests to support them. * adding the cache and tests * moved errors to var values at the top * youtube is now getting collected and tests have been setup * resolved an issue with the config * setting up the first interface, its not used yet * more updates to the cache service. Not finished yet and could see rework * added logic to monitor Youtube. Still basic logic that needs to be wired up to the database |
||
|---|---|---|
| .github/workflows | ||
| .vscode | ||
| database | ||
| domain | ||
| routes | ||
| services | ||
| .gitignore | ||
| Dockerfile | ||
| go.mod | ||
| go.sum | ||
| main.go | ||
| makefile | ||
| README.md | ||
| scheduler.go | ||
newsbot.collector.api
This is the collection service of newsbot to pull articles from the web.