powe97
|
faf303ec27
|
Add termination
|
2024-03-03 23:53:25 -05:00 |
|
powe97
|
0b97a431e1
|
Clarify steps
|
2024-03-03 18:56:33 -05:00 |
|
powe97
|
9eb5e195dc
|
Use dd instead of rsync
|
2024-03-03 17:40:25 -05:00 |
|
powe97
|
57d9f5dc4c
|
See previous commit
|
2024-03-02 19:08:06 -05:00 |
|
powe97
|
58e509d1bb
|
Clean up and potentially fix rsync
|
2024-03-02 18:35:44 -05:00 |
|
powe97
|
c3e0d558d7
|
List data folder before copying (rsync seems to not work every time?)
|
2024-03-02 16:17:11 -05:00 |
|
powe97
|
ae286917c1
|
Formatting
|
2024-03-02 02:26:58 -05:00 |
|
powe97
|
80f9ed1d95
|
Merge branch 'main' of https://github.com/quatalog/quatalog
|
2024-03-02 02:25:26 -05:00 |
|
powe97
|
bc07c559bc
|
Uh-oh
|
2024-03-02 02:22:30 -05:00 |
|
powe97
|
d5f58d8576
|
Create json dir if it doesn't already exist
|
2024-03-02 01:44:28 -05:00 |
|
powe97
|
c508798f20
|
Create courses dir if it doesn't already exist
|
2024-03-02 01:41:43 -05:00 |
|
powe97
|
30f4f49cdb
|
Fix bug where only 1 page is scraped per school and refactor
|
2024-03-01 20:32:00 -05:00 |
|
powe97
|
baa74b8ee6
|
Fix issue where only 1 page per school would get scraped properly
|
2024-03-01 18:17:53 -05:00 |
|
powe97
|
5ea6816c90
|
Fix capitalization next to smart apostrophes (really?)
|
2024-03-01 17:21:45 -05:00 |
|
powe97
|
6b5356c84f
|
Fix typo leading to bad capitalization
|
2024-03-01 15:01:20 -05:00 |
|
powe97
|
001825d3dc
|
Merge branch 'main' of https://github.com/quatalog/quatalog
|
2024-03-01 13:32:09 -05:00 |
|
powe97
|
3b608fad41
|
Fix Roman numerals issue
|
2024-03-01 13:32:02 -05:00 |
|
powe97
|
5fe4ee9f13
|
Change manually run workflow to have timeout of 2 mins
|
2024-03-01 13:12:23 -05:00 |
|
powe97
|
997d3c16a8
|
2hr timeout → 45m
|
2024-03-01 12:48:20 -05:00 |
|
powe97
|
d03be03aeb
|
Move debug print to be more accurate
|
2024-03-01 01:50:01 -05:00 |
|
powe97
|
019b777228
|
Make transfer scraper run continuously (at least as much as Github allows)
|
2024-03-01 01:45:06 -05:00 |
|
powe97
|
1a4542e20e
|
Fix crashing without timeout arg and re-add --headless
|
2024-03-01 00:29:34 -05:00 |
|
powe97
|
b0acd0e745
|
Dammit python
|
2024-02-29 22:31:09 -05:00 |
|
powe97
|
53891400ea
|
Every 15 minutes
|
2024-02-29 22:29:51 -05:00 |
|
powe97
|
c6e28d399a
|
Make timeout field have default value
|
2024-02-29 22:28:00 -05:00 |
|
powe97
|
682b1679b4
|
Run every 15 mins
|
2024-02-29 22:25:22 -05:00 |
|
powe97
|
aa4af079f8
|
Merge branch 'main' of https://github.com/quatalog/quatalog
|
2024-02-29 22:13:54 -05:00 |
|
powe97
|
cf2abf7193
|
Fix partial updates when KeyboardInterrupt happens mid-institution
|
2024-02-29 22:13:44 -05:00 |
|
powe97
|
55e34c9dd4
|
Bump versions of actions
|
2024-02-29 22:06:23 -05:00 |
|
powe97
|
efad1e9103
|
Bump versions for actions
|
2024-02-29 22:01:40 -05:00 |
|
powe97
|
cf953b2f02
|
Merge branch 'main' of https://github.com/quatalog/quatalog
|
2024-02-29 21:45:29 -05:00 |
|
powe97
|
44067261c3
|
Don't put whole repo in artifact
|
2024-02-29 21:45:22 -05:00 |
|
powe97
|
d268233d8b
|
Update transfer.yml
|
2024-02-29 21:38:17 -05:00 |
|
powe97
|
8a3e8a84d8
|
See previous commit
|
2024-02-29 21:25:53 -05:00 |
|
powe97
|
fd2da56aee
|
Make checkout data repo actually check the data repo out
|
2024-02-29 21:23:29 -05:00 |
|
powe97
|
12d844ca28
|
Fix global var fuckery
|
2024-02-29 21:21:39 -05:00 |
|
powe97
|
4916feeb19
|
Add debug timeout to workflow
|
2024-02-29 21:16:07 -05:00 |
|
powe97
|
b304e9f8d2
|
Fix scraper
|
2024-02-29 21:02:38 -05:00 |
|
powe97
|
f216c45748
|
Add if __name__ == "__main__" and fix workflow
|
2024-02-29 20:49:45 -05:00 |
|
powe97
|
15b09123ee
|
Set up workflow for transfer scraper
|
2024-02-29 20:40:15 -05:00 |
|
powe97
|
382f9080e5
|
Add transfer workflow
|
2024-02-29 17:18:50 -05:00 |
|
powe97
|
9585cc37e4
|
Add transfer scraper
|
2024-02-29 16:51:07 -05:00 |
|
powe97
|
db033af56e
|
src -> courseinfo_scraper
|
2024-02-29 16:43:38 -05:00 |
|
powe97
|
03fd5b8494
|
Disable autocompile
|
2024-02-01 23:28:41 -05:00 |
Quatalog Compiler
|
e5ebe91a5c
|
Recompile scraper : Fri Feb 2 04:28:20 UTC 2024
|
2024-02-02 04:28:20 +00:00 |
|
powe97
|
78e39968b2
|
Delete src/terms_offered.json
|
2024-02-01 23:27:55 -05:00 |
|
powe97
|
1134a8722d
|
Delete src/terms_list.json
|
2024-02-01 23:27:48 -05:00 |
|
powe97
|
e698598693
|
Delete src/prerequisites.json
|
2024-02-01 23:27:40 -05:00 |
|
powe97
|
a4bb5bdac2
|
Delete src/catalog.json
|
2024-02-01 23:27:27 -05:00 |
|
Quatalog Compiler
|
986649bade
|
Recompile scraper : Wed Aug 9 01:38:18 UTC 2023
|
2023-08-09 01:38:18 +00:00 |