Table of contents
Intercepting requests
When we browse the web, a series of HTTP requests and responses are exchanged between our browser and the pages we are visiting. There are scenarios in which it is useful to monitor or manipulate this traffic.
Request interception
Request interception enables us to observe which requests and responses are being exchanged as part of our script’s execution. For example, this is how we could print them out when we load our test website:
const { chromium } = require('playwright')
;(async () => {
const browser = await chromium.launch()
const page = await browser.newPage()
await page.setViewportSize({ width: 1200, height: 800 })
page.on('request', (request) =>
console.log('>>', request.method(), request.url())
)
page.on('response', (response) =>
console.log('<<', response.status(), response.url())
)
await page.goto('https://danube-web.shop/')
await page.screenshot({ path: 'screenshot.png' })
await browser.close()
})()
Run in Checkly
const puppeteer = require('puppeteer')
;(async () => {
const browser = await puppeteer.launch()
const page = await browser.newPage()
await page.setViewport({ width: 1200, height: 800 })
await page.setRequestInterception(true)
page.on('request', (request) => {
console.log('>>', request.method(), request.url())
request.continue()
})
page.on('response', (response) => {
console.log('<<', response.status(), response.url())
})
await page.goto('https://danube-web.shop/')
await page.screenshot({ path: 'screenshot.png' })
await browser.close()
})()
We might want to intervene and filter the outgoing requests. For example, when scraping web pages, we might want to block unnecessary elements from loading in order to speed up the procedure and lower bandwidth usage.
In the following snippet we are going to abort all requests for images on our test website. We will identify them based off of their resourceType, while letting all other requests through without modification.
const { chromium } = require('playwright')
;(async () => {
const browser = await chromium.launch()
const page = await browser.newPage()
await page.setViewportSize({ width: 1200, height: 800 })
await page.route('**/*', (route) => {
return route.request().resourceType() === 'image'
? route.abort()
: route.continue()
})
await page.goto('https://danube-web.shop/')
await page.screenshot({ path: 'screenshot.png' })
await browser.close()
})()
const puppeteer = require('puppeteer')
;(async () => {
const browser = await puppeteer.launch()
const page = await browser.newPage()
await page.setViewport({ width: 1200, height: 800 })
await page.setRequestInterception(true)
page.on('request', (request) => {
if (request.resourceType() === 'image') request.abort()
else request.continue()
})
await page.goto('https://danube-web.shop/')
await page.screenshot({ path: 'screenshot.png' })
await browser.close()
})()
As a result, you will see the website logo not being loaded.
Similarly, switching the resourceType to stylesheet would result in the target website loading without any CSS styling.
Response interception
Isolating one or more software components from their dependencies makes them easier to test. We can do so by substituting interactions with such dependencies with simulated, simplified ones. This is also known as stubbing.
Both Playwright and Puppeteer make it easy for us, as for every request we can intercept we also can stub a response.
Every time we load it, our test website is sending a request to its backend to fetch a list of best selling books. For our example, we are going to intercept this response and modify it to return a single book we define on the fly.
const { chromium } = require('playwright')
const mockResponseObject = [
{
id: 1,
title: 'How to Mock a Response',
author: 'A. Friend',
genre: 'business',
price: '0.00',
rating: '★★★★★',
stock: 65535
}
]
;(async () => {
const browser = await chromium.launch()
const page = await browser.newPage()
await page.route('https://danube-web.shop/api/books', (route) =>
route.fulfill({
contentType: 'application/json',
body: JSON.stringify(mockResponseObject)
})
)
await page.goto('https://danube-web.shop/')
await page.screenshot({ path: 'screenshot.png' })
await browser.close()
})()
const puppeteer = require('puppeteer')
const mockResponseObject = [
{
id: 1,
title: 'How to Mock a Response',
author: 'A. Friend',
genre: 'business',
price: '0.00',
rating: '★★★★★',
stock: 65535
}
]
;(async () => {
const browser = await puppeteer.launch()
const page = await browser.newPage()
await page.setRequestInterception(true)
page.on('request', (request) => {
if (request.url() === 'https://danube-web.shop/api/books') {
request.respond({
content: 'application/json',
body: JSON.stringify(mockResponseObject)
})
} else request.continue()
})
await page.setViewport({ width: 1200, height: 800 })
await page.goto('https://danube-web.shop/')
await page.screenshot({ path: 'screenshot.png' })
await browser.close()
})()
Here is what the homepage will look like with our stubbed response:
Run the above examples as follows:
node request-interception.js
Takeaways
- Playwright and Puppeteer give us control over outgoing HTTP requests.
- Playwright and Puppeteer can easily stub HTTP responses.
Further reading
- Official documentation on this topic from Playwright and Puppeteer.
- Mocks Aren’t Stubs by Martin Fowler.