Hey there,
I am trying to catch the raw request response in my step execution but I am not able to do so. What I am trying to do is retrieve raw response, parse the error field and throw it.
However, I can only catch an error, which indeed has the raw response in a message key: message: `422 - {response} I have tried parsing the response as JSON again from this key but my response contains HTML, not sure how the response is decoded in the message key of the error, but I am not able to parse it as JSON again due to the HTML in my response:
SyntaxError: Unterminated string in JSON at position 4362
Is there a way I can catch the raw response here or maybe return it again using IExecuteFunctions? Or at least disable error throwing errors based on the status code?
Here’s my code:
import {
IExecuteFunctions,
INodeExecutionData,
INodeType,
INodeTypeDescription,
NodeApiError,
} from 'n8n-workflow';
import { OptionsWithUri } from 'request';
import { DefineAPIParams } from './utils';
import { ScrapeWebPage } from './operations/ScrapeWebPage';
import { SendApiRequest } from './operations/SendApiRequest';
import { version } from '../../package.json';
export class Scrapfly implements INodeType {
description: INodeTypeDescription = {
displayName: 'Scrapfly',
name: 'Scrapfly',
icon: 'file:Scrapfly.svg',
group: ['transform'],
version: 1,
description:
'Scrapfly data collection APIs for web page scraping, screenshots, and AI data extraction',
defaults: {
name: 'Scrapfly',
},
inputs: ['main'],
outputs: ['main'],
credentials: [
{
name: 'ScrapflyApi',
required: true,
},
],
properties: [
// resources
{
displayName: 'Resource',
name: 'resource',
type: 'options',
noDataExpression: true,
options: [
{
name: 'Web Page',
value: 'WebPage',
},
{
name: 'API Request',
value: 'APIRequest',
},
{
name: 'Screenshot',
value: 'TakeScreenshot',
},
],
default: 'WebPage',
},
// operations
...ScrapeWebPage,
...SendApiRequest,
],
};
async execute(this: IExecuteFunctions): Promise<INodeExecutionData[][]> {
const items = this.getInputData();
let responseData;
const returnData = [];
const resource = this.getNodeParameter('resource', 0) as string;
// TODO: QA error handling and change returned fields on errors
for (let i = 0; i < items.length; i++) {
const params = DefineAPIParams.call(this, i);
// set the method to GET by default, update when scraping
let method = 'GET';
if (resource === 'APIRequest' || resource === 'WebPage') {
method = this.getNodeParameter('method', i) as string;
const options: OptionsWithUri = {
headers: {
Accept: 'application/json',
'User-Agent': `n8n integration v${version}`,
},
method: method,
uri: `https://api.scrapfly.io/scrape?${params.toString()}`,
json: true,
};
// add request body if the method is POST, PUT, or PATCH
if (['POST', 'PUT', 'PATCH'].includes(method) && params.has('body')) {
options.body = params.get('body');
params.delete('body');
}
try {
responseData = await this.helpers.requestWithAuthentication.call(
this,
'ScrapflyApi',
options,
)
returnData.push(responseData);
} catch (e) {
throw new NodeApiError(this.getNode(), e, {
httpCode: e.httpCode,
description: `[${e.errorCode}] ${e.description}`,
});
}
}
}
return [this.helpers.returnJsonArray(returnData)];
}
}
Information on my n8n setup
- n8n version:1.60.1
- Database (default: SQLite):
- n8n EXECUTIONS_PROCESS setting (default: own, main)
- Running n8n via npm
- Operating system: win11
