Still not able to scrape Amazon.in [Login wall]

[guptasarthak10]

Still not able to scrape Amazon links.

My pipeline: https://www.gumloop.com/pipeline?workbook_id=hyaVfiarKET4bBHQ3Yf4Jx&run_id=gKEuipmScPu9Sf8mciagTd

Latest run with the below output from the Amazon reviews node: https://www.gumloop.com/pipeline?run_id=gKEuipmScPu9Sf8mciagTd&workbook_id=hyaVfiarKET4bBHQ3Yf4Jx

Amazon link I’m trying to scrape: MyMuse Groove Full Body Electric Stress Relief Massager, Medical Grade Silicone, Flexible Neck, Lemon Tart, Ergonomic Handle, Waterproof & Rechargeable, 10 Speeds : Amazon.in: Health & Personal Care

ASIN: B0BXXXYVX2 (Being scrapped properly. Failing after this step)

[Gumloop_Bot]

Hey @guptasarthak10! If you’re reporting an issue with a flow or an error in a run, please include the run link and make sure it’s shareable so we can take a look.

1. Find your run link on the history page. Format: https://www.gumloop.com/pipeline?run_id={{your_run_id}}&workbook_id={{workbook_id}}

2. Make it shareable by clicking “Share” → ‘Anyone with the link can view’ in the top-left corner of the flow screen.
   

3. Provide details about the issue—more context helps us troubleshoot faster.

You can find your run history here: https://www.gumloop.com/history

[Wasay-Gumloop]

Hey @guptasarthak10 – You should feed the scraper the link in this format: MyMuse Groove Full Body Electric Stress Relief Massager, Medical Grade Silicone, Flexible Neck, Lemon Tart, Ergonomic Handle, Waterproof & Rechargeable, 10 Speeds : Amazon.in: Health & Personal Care

The full amazon link would work since that doesn’t require logging in. If you try to view this link it would prompt you to sign-in: https://www.amazon.in/product-reviews/B0CLCLYJN1

[guptasarthak10]

@Wasay-Gumloop

The full amazon link would work since that doesn’t require logging

I don’t get it. Both the links are the same. Which URL format should I use?

[Wasay-Gumloop]

You should use the full URL format from the doc, currently the link you’re passing to the node is this: https://www.amazon.in/product-reviews/B0CLCLYJN1 - if you open this in incognito it’ll prompt you to sign-in.

[guptasarthak10]

Latest run: https://www.gumloop.com/pipeline?run_id=M37Auu9c32jg3d5zFanzZd&workbook_id=hyaVfiarKET4bBHQ3Yf4Jx

Still getting an error. Seems like the Amazon scraper node is not working. On opening the entire link on my private browser, I’m able to view the Amazon page. But failing after the scrapping node.

[Wasay-Gumloop]

Yeah, it’s always a bit of a cat-and-mouse chase trying to get past blockers with sites like Amazon. Adding a wait function and enabling advanced scraping should help: https://www.gumloop.com/pipeline?workbook_id=7zifVSyrz3ZL1poyyeoEpa&tab=1&run_id=biaZhRbRy2Yoe5ZhNM5Na4