Web Scraping/Data Mining Internship (Remote)

Selected intern's day-to-day responsibilities include:

1. Work on crawling, extracting, and processing data (e.g Scrapy, pandas, MapReduce, SQL, BeautifulSoup, etc.)
2. Gather and process raw data at scale (including writing scripts, web scraping, calling/creating APIs, etc.) from the web/internet
3. Develop the capability to efficiently scrape data from the web from multiple sources
4. Scrape difficult websites by deploying anti-blocking and anti-captcha tools
5. Develop various RestFul APIs & Integrate them with various data sources
6. Develop tools & techniques related to data extraction from web or pdf files and other process automation
7. Develop frameworks for automating and maintaining a constant flow of data from multiple sources
8. Optimize the scraping capability to ensure the data is scrapped efficiently with the minimum usage of server bandwidth

2