phantop/quatalog-scraper
1
0
Fork 0
mirror of https://github.com/quatalog/quatalog.git synced 2024-11-26 16:52:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' of https://github.com/quatalog/quatalog

Browse source
This commit is contained in:
powe97 2024-03-02 02:22:39 -05:00
parent bc07c559bc d5f58d8576
commit 80f9ed1d95
No known key found for this signature in database
GPG key ID: 7D1663B10978D1BA
2 changed files with 2 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
.github/workflows/scraper.yml vendored
View file

@ -110,6 +110,7 @@ jobs:
- name: Merge data
run: |
mkdir -p quatalog-site/courses quatalog-site/json
rsync -avz courses/ quatalog-site/courses/
rsync -avz searchable_catalog.json quatalog-site/json/searchable_catalog.json
rsync -avz courses_list.json quatalog-site/json/courses_list.json

2
transfer_scraper/main.py
View file

@ -169,7 +169,7 @@ def main():
"https://tes.collegesource.com/publicview/TES_publicview01.aspx?rid=f080a477-bff8-46df-a5b2-25e9affdd4ed&aid=27b576bb-cd07-4e57-84d0-37475fde70ce"
)
print(f'Title is {driver.find_element(By.TAG_NAME, "title").get_attribute("innerText")}')
print(f'Title is {driver.find_element(By.TAG_NAME, "title").get_attribute("innerText")}', file=sys.stderr)
num_pages = int(
driver.find_element("id", "lblInstWithEQPaginationInfo").text.split()[-1]