Question about htaccess, robots.txt and crawlers.

[dsarosh]

Hello,
I have a couple of questions.

1) How long after I register a domain can I expect the search engines index my site?

2) I do not have any robots.txt or any htaccess files on my server. Is that okay? Are they required for search engines to crawl my site?

Thanks,
Sarosh.

 

[Hacker]

Did you submit your site to google.

 

[dsarosh]

Not yet because it is still almost empty.

But I wanted to know how long it would take to get indexed if:

1) I submitted the site to search engines

2) I did not submit it to sites

3) I do not use any robots or htaccess file (I have learned that robots.txt and htaccess files are not required to get the site indexed)

 

[rajkumar_pb]

Hello,
I have a couple of questions.

1) How long after I register a domain can I expect the search engines index my site?

Once you submitted your site to google, it'll start the job within a minimum of 24 hours to three days.

2) I do not have any robots.txt or any htaccess files on my server. Is that okay? Are they required for search engines to crawl my site?

Thanks,
Sarosh.

I think creating robots.txt is not a big deal, so create it and upload it. Simple. No need for htaccess as i guess you're on shared server.

 

[Malique]

robots.txt are used by crawlers, it contains paths which "ideally" the crawlers should not read/access for caching/indexing. this text file is optional and following the robots.txt is upto the crawlers which access your site.