
Building a link-in-bio and short-link site with EmDash and Astro
How I built rser.me: a link-in-bio page and short-link service on EmDash, Astro and Cloudflare Workers, with link cards that double as short links and 302 redirects.
Read moreJust as you can use robots.txt to block search engines and various bots from crawling your site, you can do the same with OpenAI's GPTBot.
GPTBot is OpenAI’s web crawler and can be identified by the following user agent and string.
User agent token: GPTBot
Full user-agent string: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.0; +https://openai.com/gptbot)To block GPTBot from crawling your site entirely, you can add the following to your robots.txt file:
User-agent: GPTBot
Disallow: /If you want to allow GTPBot to crawl certain areas of your site, but block it from other areas, you can do this:
User-agent: GPTBot
Allow: /directory-1/
Disallow: /directory-2/
How I built rser.me: a link-in-bio page and short-link service on EmDash, Astro and Cloudflare Workers, with link cards that double as short links and 302 redirects.
Read more
How I set up my Xteink X4 Pro e-reader: CrossPoint Reader firmware flashed from the browser, custom sleep-screen wallpapers, and books and PDFs over Wi-Fi.
Read more
How Talk To My Sites sends signups, logins and funnel events to a Get Butters feed, and how I alert on the errors that actually matter.
Read more