lewisdale.dev/src/robots.txt

User-Agent: Bytespider
Disallow: /

User-agent: GPTBot
Disallow: /

User-Agent: *
Allow: /
Disallow: /assets/
Disallow: /wp-content/
Host: https://lewisdale.dev
Add robots.txt 2023-05-11 09:30:28 +00:00			`User-Agent: Bytespider`
			`Disallow: /`

Ban ChatGPT from scraping me 2023-08-25 12:44:00 +00:00			`User-agent: GPTBot`
			`Disallow: /`

Add robots.txt 2023-05-11 09:30:28 +00:00			`User-Agent: *`
			`Allow: /`
			`Disallow: /assets/`
Add robots.txt 2023-05-11 09:30:52 +00:00			`Disallow: /wp-content/`
Add robots.txt 2023-05-11 09:30:28 +00:00			`Host: https://lewisdale.dev`