phantop/quatalog-scraper
1
0
Fork 0
mirror of https://github.com/quatalog/quatalog.git synced 2024-11-20 05:43:49 +00:00
Code Issues Packages Projects Releases Wiki Activity
313 commits 1 branch 0 tags 6.4 MiB
Commit graph

98 commits

Author SHA1 Message Date
powe97 0109006d12
Nevermind 2024-03-17 20:07:48 -04:00
powe97 cae1b09981
Add zero padding 2024-03-17 20:05:07 -04:00
powe97 6de711b23e
Update transfer.yml 2024-03-17 10:38:39 -04:00
powe97 99ba8873bd
Run twice a day 2024-03-16 21:25:42 -04:00
powe97 eb77e84535
Fix artifact upload 2024-03-16 21:23:23 -04:00
powe97 ecfb176c46
Create scrape_page.yml 2024-03-16 20:46:25 -04:00
powe97 ea09d33ac2
Convert to use reusable workflow 2024-03-16 20:45:57 -04:00
powe97 f7c5c2461a
Update transfer.yml 2024-03-16 13:07:22 -04:00
powe97 130dca8b89
Oops 2024-03-16 12:55:53 -04:00
powe97 1403b42a5c
Oops 2024-03-16 12:43:11 -04:00
powe97 463a2aac03
Create generate-csvs.yml 2024-03-16 12:40:14 -04:00
powe97 d0f2f969d2
Run CSV generator 2024-03-16 12:19:01 -04:00
powe97 de7360f1c7
Convert JSON to by-course format 2024-03-16 01:28:33 -04:00
powe97 1fa7ab61af
Rename transfer scraper 2024-03-16 00:32:59 -04:00
powe97 45cfba68ac
Sort output JSON 2024-03-06 12:10:23 -06:00
powe97 aec272e28e
Run once a day 2024-03-06 02:43:41 -06:00
powe97 c21dec05ad
Re-enable fail-fast 2024-03-06 02:36:33 -06:00
powe97 6a1395c054
Fix workflow not being able to commit and remove tee 2024-03-06 02:05:49 -06:00
powe97 0007bde18a
Recombine JSONs 2024-03-06 00:43:26 -06:00
powe97 56c9268398
Disable fail-fast 2024-03-05 20:49:08 -05:00
powe97 e45318404d
Update transfer.yml 2024-03-05 19:06:49 -05:00
powe97 10715c89e3
Update transfer.yml 2024-03-05 19:05:41 -05:00
powe97 42dbf3c19a
Update transfer.yml 2024-03-05 18:46:02 -05:00
powe97 985f40c4e7
Set up matrix jobs 2024-03-05 18:42:05 -05:00
powe97 acdd08168f
Update transfer.yml 2024-03-05 18:27:51 -05:00
powe97 0b97a431e1
Clarify steps 2024-03-03 18:56:33 -05:00
powe97 9eb5e195dc
Use dd instead of rsync 2024-03-03 17:40:25 -05:00
powe97 57d9f5dc4c
See previous commit 2024-03-02 19:08:06 -05:00
powe97 58e509d1bb
Clean up and potentially fix rsync 2024-03-02 18:35:44 -05:00
powe97 c3e0d558d7
List data folder before copying (rsync seems to not work every time?) 2024-03-02 16:17:11 -05:00
powe97 d5f58d8576
Create json dir if it doesn't already exist 2024-03-02 01:44:28 -05:00
powe97 c508798f20
Create courses dir if it doesn't already exist 2024-03-02 01:41:43 -05:00
powe97 5fe4ee9f13
Change manually run workflow to have timeout of 2 mins 2024-03-01 13:12:23 -05:00
powe97 997d3c16a8
2hr timeout → 45m 2024-03-01 12:48:20 -05:00
powe97 019b777228
Make transfer scraper run continuously (at least as much as Github allows) 2024-03-01 01:45:06 -05:00
powe97 53891400ea
Every 15 minutes 2024-02-29 22:29:51 -05:00
powe97 682b1679b4
Run every 15 mins 2024-02-29 22:25:22 -05:00
powe97 55e34c9dd4
Bump versions of actions 2024-02-29 22:06:23 -05:00
powe97 efad1e9103
Bump versions for actions 2024-02-29 22:01:40 -05:00
powe97 cf953b2f02
Merge branch 'main' of https://github.com/quatalog/quatalog 2024-02-29 21:45:29 -05:00
powe97 44067261c3
Don't put whole repo in artifact 2024-02-29 21:45:22 -05:00
powe97 d268233d8b
Update transfer.yml 2024-02-29 21:38:17 -05:00
powe97 fd2da56aee
Make checkout data repo actually check the data repo out 2024-02-29 21:23:29 -05:00
powe97 4916feeb19
Add debug timeout to workflow 2024-02-29 21:16:07 -05:00
powe97 f216c45748
Add if __name__ == "__main__" and fix workflow 2024-02-29 20:49:45 -05:00
powe97 15b09123ee
Set up workflow for transfer scraper 2024-02-29 20:40:15 -05:00
powe97 382f9080e5
Add transfer workflow 2024-02-29 17:18:50 -05:00
powe97 03fd5b8494
Disable autocompile 2024-02-01 23:28:41 -05:00
powe97 9ec4198155
add header to csv 2023-07-29 17:17:48 -04:00
powe97 9b877abe66
Fix typo 2023-07-29 17:10:27 -04:00