Attendance Spider¶

class scraper.spiders.attendance_spider.AttendanceSpider(username, password, chatID, *args, **kwargs)¶

Scrape attendance figures from http://report.aldel.org/student/attendance_report.php and store the figures in database with scraper.pipelines.LecturePipeline and scraper.pipelines.PracticalPipeline

Parameters:	InitSpider (Spider) – Base Spider with initialization facilities

name = 'attendance'¶

allowed_domains = ['report.aldel.org']¶

login_page = 'http://report.aldel.org/student_page.php'¶

start_urls = ['http://report.aldel.org/student/attendance_report.php']¶

init_request()¶: This function is called before crawling starts.

login(response)¶: Generate a login request.

check_login_response(response)¶: Check the response returned by a login request to see if we are successfully logged in.

parse(response)¶: Send a SplashRequest and forward the response to parse_result()

parse_result(response)¶

Downloads and saves the attendance report in files/<Student_ID>_attendance.png format.

Also scrapes every attendance record from the webpage and passes it to LecturePipeline and PracticalPipeline.

scraper.spiders.attendance_spider.scrape_attendance(username, password, chatID)¶

Run the spider multiple times, without hitting ReactorNotRestartable exception. Forks own process.

Parameters:	username (str) – student’s PID (format: XXXNameXXXX) where X - integers password (str) – student’s password for student portal chatID (str) – 9-Digit unique user ID