In case you have a little Internet site (underneath ten internet pages), then I wouldn’t worry too much about possibly of these files. On the other hand, when you’re making use of WordPress, You should utilize a plugin like Yoast Search engine optimisation to generate the two a robots.txt https://www.youtube.com/watch?v=EVeTP0aV8gY