What browsing with ChatGPT taught me

Photo by Marc Rafanell López on Unsplash — yeah, this is another type of crawler

There are websites out there which aren’t ready for the primitive browsing ChatGPT affords

ChatGPT plugins are how GPT-4, the LLM backing ChatGPT, pulls in information from the real world into its inferences.

If you are a ChatGPT Plus subscriber, you should have a special plugin which browses urls you supply during a chat session to add more context to the conversation.

Here’s the thing about this browser plugin.

It is essentially a headless browser which uses Selenium to interact with the given url.

Selenium, great as it is, has a limitation. It is not Javascript run time.

This means that if your page is only rendered after the client has downloaded and executed a bunch of javascript, ChatGPT is not going to be able to read your website.

You will end up with errors like this:

ChatGPT failed to open and process an SPA

This is especially problematic because a lot of modern websites are Single Page Apps or a hybrid of static content hydrated with live data pulled by javascript running inside the browser.

How come this does not affect…