Dust Community Icon
Join the Community
Help

Retrieving Data from Public Notion Pages: Crawler Challenges Explained

Avatar of Yoshihito ImaiYoshihito Imai
·
·

Hi 👋 Is it possible to retrieve information from publicly shared Notion pages using a website crawler? When I try, I get an "Empty content" error.

7 comments

  • Avatar of Remi
    Remi
    ·
    ·

    Thanks for the question Yoshihito Imai. It's strange, it doesn't work on my side either ! Let me ask the team 👀

  • Avatar of Yoshihito Imai
    Yoshihito Imai
    ·
    ·

    Thank you for your response🙌 As shown in the screenshot, an "empty content" error is being displayed. This same issue is occurring with other published Notion pages as well. Am I making a mistake in my settings somewhere?

  • Avatar of Remi
    Remi
    ·
    ·

    Thanks for the clarifications! I tried it on my side and it didn't work either. I asked the eng team 👍

    ✨1
  • Avatar of Yoshihito Imai
    Yoshihito Imai
    ·
    ·

    Remi Have you made any progress on this case?

  • Avatar of Yoshihito Imai
    Yoshihito Imai
    ·
    ·

    Remi Just a friendly reminder🙌

  • Avatar of Remi
    Remi
    ·
    ·

    Apologies Yoshihito Imai for letting this one slip. Unfortunately, it looks like it is a known limitation of the "website" feature - some sites can't easily be scrapped for a bunch of different reasons. The best in your case would probably to manually copy/paste the content in documents in a Dust folder. It is not ideal I know 😬