Commit Graph

153 Commits

Author SHA1 Message Date
James Tombleson 0d61cdf268 brought over the same config from the portal to push to hub.docker.com
continuous-integration/drone/pr Build is passing Details
2023-06-17 10:25:31 -07:00
jtom38 1257e8b166 Merge pull request 'pulling pat from drone' (#6) from features/ci-update-auth into master
continuous-integration/drone/push Build is failing Details
Reviewed-on: #6
2023-06-13 22:28:07 -07:00
James Tombleson c36dad769c pulling pat from drone
continuous-integration/drone/pr Build is passing Details
2023-06-13 22:26:28 -07:00
Grudging0288 a89162d130 Merge pull request 'attempting upload to gitea' (#4) from features/ci-upload-to-gitea into master
continuous-integration/drone/push Build is failing Details
Reviewed-on: #4
2023-06-13 22:12:28 -07:00
James Tombleson 1e8c489df7 attempting upload to gitea
continuous-integration/drone/pr Build is passing Details
2023-06-13 22:09:47 -07:00
jtom38 43cd57a158 Merge pull request 'You can access /health without a api key now.' (#3) from features/health-no-auth into master
continuous-integration/drone/push Build is passing Details
Reviewed-on: #3
2023-06-13 21:39:57 -07:00
James Tombleson 63a5ce9947 attempting to not build docker on pr
continuous-integration/drone/pr Build is passing Details
2023-06-13 21:37:32 -07:00
James Tombleson a3b1904923 You can access /health without a api key now.
continuous-integration/drone/pr Build is passing Details
2023-06-13 21:32:49 -07:00
jtom38 db6f140858 Merge pull request 'Adding back drone ci, went missing' (#2) from features/ci/add-drone-ci into master
continuous-integration/drone Build is passing Details
Reviewed-on: #2
2023-06-12 17:38:07 -07:00
James Tombleson 6805f0c9a0 Adding back drone ci, went missing 2023-06-12 17:37:40 -07:00
jtom38 18c3fac57f Merge pull request 'removing github ci' (#1) from features/remove-github-ci into master
Reviewed-on: #1
2023-06-12 17:35:33 -07:00
James Tombleson 5415e957c0 removing github ci 2023-06-12 17:35:04 -07:00
James Tombleson e4c5d1be29 code was added to allow multiple api keys to talk to the api. 2023-06-12 15:15:04 -07:00
James Tombleson 39bf92dd53
Features/codeprojects/disable commit checks (#30)
* hard disable for checking commits till I can debug it more

* New HtmlCode method added
2023-05-25 21:42:32 -07:00
James Tombleson 70440aa3f5
Features/subscription details dto (#29)
* DTO was updated to reflect the new options

* SubscriptionsController.cs added id's as part of the path not query

* Refactored DiscordNotificationJob.cs to break apart the nesting

* Added a test to make sure commits would not be sent based on model values
2023-04-14 21:39:02 -07:00
James Tombleson 84b4137bdd
Features/codeproject/subscription options (#28)
* Updated migrations to add new columns to subscriptions

* repos updated with new columns

* dto updated with new columns

* subscription model was updated

* DiscordNotificationJob.cs was updated to reflect subscription options

* updated seed for codeproject subscriptions
2023-04-13 22:13:06 -07:00
James Tombleson 02c94d442e
Breaking up a migration because it was a mistake to cobble them together (#27) 2023-04-11 18:31:56 -07:00
James Tombleson 117653c001
Features/add code projects watcher (#26)
* Migration added to update Articles to define if Release or Commit

* CodeProjectWatcher Job was created from GithubWatcher as this will target services like gitlab and also gitea.

* article model was updated to reflect migration changes

* Added CodeProjects to startup

* Seed was updated with CodeProjects and some new defaults

* Added Delete call for Sources

* Added a route to cleanup all records based on SourceId

* Added CodeProject const values to load from config

* minor changes to the rss controller

* Added codeprojects to the routes to trigger the job
2023-04-10 22:59:13 -07:00
James Tombleson a25c44d8cc
Features/api/youtube source (#25)
* corrected the youtube url getting added wrong

* updated seed
2023-04-08 10:10:08 -07:00
James Tombleson bacc04ad7d
Features/enabling youtube (#24)
* Adding a youtube controller to trigger the job

* Renamed jobs controller to rss

* cleaned up background jobs and added youtube to the startup.

* Handled merge issues and validated things are still working.
2023-04-08 09:30:59 -07:00
James Tombleson 4b6ba97a87
fixed code to allow compile (#23) 2023-04-03 09:32:42 -07:00
James Tombleson c0251e2485
Features/discord errors (#22)
* catching another http error code.

* more discord logging

* Setting min logging to debug for now.

* program logging is set to info but job logging is debug.
2023-04-03 09:31:00 -07:00
James Tombleson 33bf88d866
Features/discord errors (#21)
* catching another http error code.

* more discord logging

* Setting min logging to debug for now.
2023-04-03 08:57:19 -07:00
James Tombleson af2f700e41
Features/discord errors (#20)
* catching another http error code.

* more discord logging
2023-04-03 06:46:35 -07:00
James Tombleson d23019926b
catching another http error code. (#19) 2023-04-02 08:03:59 -07:00
James Tombleson f1f0e50098
Features/discord logging (#18)
* Had the wrong flag for rss enabled

* Adding more logging for discord.
2023-04-01 16:22:44 -07:00
James Tombleson 1bc9b8d842
expanded logging on DiscordNotificationJob.cs (#17) 2023-04-01 09:24:26 -07:00
James Tombleson 09fe274752
Features/less comples config managment (#16)
* New single const file for all config info

* seed was updated to reflect route params

* Adjusting config loads to make it easier to follow

* test updates for config loading
2023-04-01 08:53:34 -07:00
James Tombleson b7b50f5124
updated path for migrations (#15) 2023-03-31 23:13:35 -07:00
James Tombleson adb4799206
Features/missing files (#14)
* Added jobs Controller to trigger collection.

* Added backgroundjobs to move them out of program.cs

* new column to track youtube ID values and adding a sourceid column on the icon for linking

* Added icon table repo

* added interface for IconsRepo

* hey the missing config models

* adding section const keys to pull blocks of configs

* Added youtubewatcher to the code but not ready to enable it in the background.  More testing needed.

* Test... improvements?
2023-03-31 23:00:15 -07:00
James Tombleson 9be985da0a
Features/adding youtube (#13)
* Found the meta tags on youtube... in the body and updated the client to pull them out.

* Updated namespace on test

* I think formatting cleaned this up

* Seed migrations have been cleaned up to get my configs out and moving them to a script.

* Updates to the ISourcesRepository.cs to allow for new calls to the db.

* formatter

* Db models updated. Icon now can track sourceID and source can have a youtube id.

* Updated api logger to ignore otel if no connection string given.

* updated docker init so I can run migrations from the image

* seed was updated to reflect the new api changes

* Updated the SourcesController.cs to grab icon data.

* Added reddit const values

* Minor changes to HtmlPageReader.cs

* Jobs are now pulling in the config section to bundle values.

* Removed youtube api, not needed anymore.

* test updates
2023-03-31 22:49:39 -07:00
James Tombleson ac6bdaa184
Added otel support and pushed data to my server. (#12) 2023-03-19 22:54:17 -07:00
James Tombleson 607df99bfb
the health ui shows more details now (#11) 2023-03-14 07:21:34 -07:00
James Tombleson 9425348974
Features/add health (#10)
* updated the api to have a basic health check

* docker-compose.yaml was updated to add health check
2023-03-13 22:25:45 -07:00
James Tombleson b4768b7cff removing DiscordWebhookClient repo as I changed my mind on porting the code currently. 2023-03-13 22:05:49 -07:00
James Tombleson 2bea52536c Initial seed script is now working 2023-03-13 22:00:26 -07:00
James Tombleson 716e492311 ignoring .env files for docker compose secrets 2023-03-13 22:00:03 -07:00
James Tombleson b8ec9ed5f2 updated docker-compose.yaml to enable variable loading from a env file 2023-03-13 21:59:39 -07:00
James Tombleson af5937d1ef updated makefile to support migrations via docker 2023-03-13 21:59:03 -07:00
James Tombleson dfd77a1253 expanded on error handing when a record did not exist 2023-03-13 21:58:33 -07:00
James Tombleson 578197cb75 Added better error checking on the new route requests 2023-03-13 21:57:32 -07:00
James Tombleson 799668a059
Features/pulling GitHub (#9)
* Still working though it but looking good on releases

* added the example discord message test

* updated source repo to return an existing record before a new is added

* updated the sources repo interface

* updated new routes to check for existing records

* starting to migrate the seed out of the sql migrations

* A new seed script was made to reload the db from the api

* Docker image works locally

* Adding CI to build docker image

* ... disabled swagger so I can test docker

* Added more to the github job but its not finished.  Isnt pulling sources yet.

* cleaned up formatting

* Controller updates to look for existing records when requesting a new one

* null check cleanup

* namespace fix
2023-03-11 10:43:06 -08:00
James Tombleson aa53b1eeeb
Features/testing discord queue (#8)
* new seed and corrected some issues talking to the db

* adding some prep work for adding youtube back again

* small updates for rss and getting results from discord job

* new command to reload db

* added discord alerts job to hangfire
2023-03-05 22:33:41 -08:00
James Tombleson bb832ed441
New custom discord webhook client (#7) 2023-03-05 20:12:59 -08:00
James Tombleson 521940ca4f
Features/more rss improvements (#6)
* exposing connectionStrings to controllers

* First controller added to start testing

* corrected param to be page not age

* new model to map connection strings to for the controllers

* HelloWorldJob uses options now to make hangfire happy

* improved the html reader to find some rss feeds and start to extract the body of the content

* moved html parser to its own namespace and make a sub client to process theh header

* helpful vsc changes

* updated rss watcher to include the sourceId so it can be added to the db call

* updated tests to reflect changes

* updated gitignore to avoid trash and moved over my makefile

* More routes and added serilog

* adding more database calls for the controllers

* Updated interfaces for the tables

* Added Serilog to jobs

* removed default files

* Added more routes and added DTO

* Added DTO objects and SourceType Consts for easy usage

* updated discord model name to follow the pattern

* updated formatting

* new dto objects and Subscriptions repo interface

* added subscription db and api calls

* focusing on the twitter tags as most sites focus on them

* updated test to pull a html based feed
2023-02-26 09:40:04 -08:00
James Tombleson 17e97b4e09
Features/table clients (#5)
* inital table clients are almost ready

* adding new repo interfaces

* adding MS DI to the tables

* adding tables and MS DI to jobs

* updated how the config builder works and pass Iconfig to jobs

* Updated how articles interface returns a value

* updated constructors to support DI and removed static call

* added source consts and model notes

* updated sources.type to contain the value to link what job collects it

* added RssWatcherJob to hangfire

* DI and hangfire jobs wont work.  Defering to options classes

* Services updated to have options exposed over DI

* Tests have been updated.. more to come
2023-02-19 21:39:03 -08:00
James Tombleson 9f3a6323a6
Features/html meta extractor (#4)
* gave api access to the db project

* added db models

* working on rss extraction and meta extraction

* test project to debug rsswatcherjob

* added new configs for the project

* new interface to define collectors

* basic rss extraction and article details are now exposed

* tests updated for rss pull

* starting to get dapper working.  Query works but insert seems to have a value issue

* removed dapper from services

* added some basic tests for db calls
2023-02-16 22:19:05 -08:00
James Tombleson 9f5772551c
updated domain project with db models (#3) 2023-02-15 21:00:22 -08:00
James Tombleson 549953ca61
Merge pull request #2 from jtom38/features/database-project 2023-02-15 20:49:33 -08:00
James Tombleson 0ae9c9afb2 added the database project and migrated models and migrations 2023-02-15 20:47:21 -08:00