phantop/quatalog-scraper
1
0
Fork 0
mirror of https://github.com/quatalog/quatalog.git synced 2024-11-08 08:04:24 +00:00
Code Issues Packages Projects Releases Wiki Activity
232 commits 1 branch 0 tags 6.4 MiB
Commit graph

232 commits

This branch
This branch
All branches
Author SHA1 Message Date
powe97 019b777228
Make transfer scraper run continuously (at least as much as Github allows) 2024-03-01 01:45:06 -05:00
powe97 1a4542e20e
Fix crashing without timeout arg and re-add --headless 2024-03-01 00:29:34 -05:00
powe97 b0acd0e745
Dammit python 2024-02-29 22:31:09 -05:00
powe97 53891400ea
Every 15 minutes 2024-02-29 22:29:51 -05:00
powe97 c6e28d399a
Make timeout field have default value 2024-02-29 22:28:00 -05:00
powe97 682b1679b4
Run every 15 mins 2024-02-29 22:25:22 -05:00
powe97 aa4af079f8
Merge branch 'main' of https://github.com/quatalog/quatalog 2024-02-29 22:13:54 -05:00
powe97 cf2abf7193
Fix partial updates when KeyboardInterrupt happens mid-institution 2024-02-29 22:13:44 -05:00
powe97 55e34c9dd4
Bump versions of actions 2024-02-29 22:06:23 -05:00
powe97 efad1e9103
Bump versions for actions 2024-02-29 22:01:40 -05:00
powe97 cf953b2f02
Merge branch 'main' of https://github.com/quatalog/quatalog 2024-02-29 21:45:29 -05:00
powe97 44067261c3
Don't put whole repo in artifact 2024-02-29 21:45:22 -05:00
powe97 d268233d8b
Update transfer.yml 2024-02-29 21:38:17 -05:00
powe97 8a3e8a84d8
See previous commit 2024-02-29 21:25:53 -05:00
powe97 fd2da56aee
Make checkout data repo actually check the data repo out 2024-02-29 21:23:29 -05:00
powe97 12d844ca28
Fix global var fuckery 2024-02-29 21:21:39 -05:00
powe97 4916feeb19
Add debug timeout to workflow 2024-02-29 21:16:07 -05:00
powe97 b304e9f8d2
Fix scraper 2024-02-29 21:02:38 -05:00
powe97 f216c45748
Add if __name__ == "__main__" and fix workflow 2024-02-29 20:49:45 -05:00
powe97 15b09123ee
Set up workflow for transfer scraper 2024-02-29 20:40:15 -05:00
powe97 382f9080e5
Add transfer workflow 2024-02-29 17:18:50 -05:00
powe97 9585cc37e4
Add transfer scraper 2024-02-29 16:51:07 -05:00
powe97 db033af56e
src -> courseinfo_scraper 2024-02-29 16:43:38 -05:00
powe97 03fd5b8494
Disable autocompile 2024-02-01 23:28:41 -05:00
Quatalog Compiler e5ebe91a5c Recompile scraper : Fri Feb 2 04:28:20 UTC 2024 2024-02-02 04:28:20 +00:00
powe97 78e39968b2
Delete src/terms_offered.json 2024-02-01 23:27:55 -05:00
powe97 1134a8722d
Delete src/terms_list.json 2024-02-01 23:27:48 -05:00
powe97 e698598693
Delete src/prerequisites.json 2024-02-01 23:27:40 -05:00
powe97 a4bb5bdac2
Delete src/catalog.json 2024-02-01 23:27:27 -05:00
Quatalog Compiler 986649bade Recompile scraper : Wed Aug 9 01:38:18 UTC 2023 2023-08-09 01:38:18 +00:00
powe97 b81968e43a
Change to an ordered set so that the courses_list doesn't change order every rescrape 2023-08-08 21:37:19 -04:00
powe97 9ec4198155
add header to csv 2023-07-29 17:17:48 -04:00
powe97 9b877abe66
Fix typo 2023-07-29 17:10:27 -04:00
powe97 18ccae5fa7
Issue with relative paths 2023-07-27 23:58:34 -04:00
powe97 4601554c9a
Add Kuzmin CSV file 2023-07-27 23:54:26 -04:00
Quatalog Compiler b598a039a5 Recompile scraper : Thu Jul 27 23:29:58 UTC 2023 2023-07-27 23:29:58 +00:00
powe97 f6339a6d94 Use the fix course IDs function to fix course IDs in course pills 2023-07-27 19:29:16 -04:00
powe97 f3801ba31c Partial rewrite getting course title/description 2023-07-27 19:04:34 -04:00
powe97 6bdd4f6944 Curse whoever came up with the INQR code 2023-07-27 18:24:55 -04:00
powe97 15ea56b1f7 Ok nvm this is unfixable until the new catalog is out 2023-07-27 18:22:36 -04:00
powe97 74c6b418ed Catalog entry moment 2023-07-27 18:19:07 -04:00
powe97 00806bbe60 Yet more fixes 2023-07-27 18:16:40 -04:00
powe97 779b596d63 IHSS -> INQR (why) 2023-07-27 18:12:44 -04:00
powe97 f9a10a59dd fix 2023-07-27 17:36:17 -04:00
powe97 3aa7aaa509 Why do I do this to myself 2023-07-27 17:29:55 -04:00
powe97 098c89a31c ITEC -> ITWS 2023-07-27 17:17:30 -04:00
Quatalog Compiler 58c7f1beeb Recompile scraper : Sat Jul 1 18:30:23 UTC 2023 2023-07-01 18:30:23 +00:00
powe97 f575c78776 Fix STSO/S/H discrepancy in SIS links 2023-07-01 14:27:07 -04:00
powe97 840e60b1b5 Fix yet another STSO bug... how did this go unnoticed 2023-07-01 14:28:41 -04:00
Quatalog Compiler 1ef053a483 Recompile scraper : Tue Jun 27 03:26:54 UTC 2023 2023-06-27 03:26:54 +00:00