skip to content

Web Scraping

Marketing scholars increasingly use web scraping and application programming interfaces (APIs) to collect data from the internet. Yet, despite the widespread use of such web data, the idiosyncratic and sometimes insidious challenges in its collection have received limited attention. This workshop presents a methodological framework outlining how addressing validity concerns requires the joint consideration of idiosyncratic technical and ethical questions along the three stages of collecting web data: selecting data sources, designing the data collection, and extracting the data. To facilitate future applications in marketing research, the session will also spotlight underutilized web sources and explain how researchers can use them to improve measures, increase the diversity of their research, overcome limitations of other methods, and study emerging marketing phenomena and trends.