Is A Php File Searchable?

rockarolla

Feb 13 2008, 03:17 AM

Hi,

I have made my web site solely stored in a SQL database...that will say if I need to load a page I take it our from the data base and then display it.

My question is: is any searcg engine able to ``crawl'' into my web site content?

I would appreciate some info so that I can change the way my web works.

Comment/Reply (w/o sign-up)

Sten

Feb 13 2008, 03:29 AM

Provided this is what you mean, yes.

So I gather you've stored everything in a database and using PHP to get a part of it and show it, just like a CMS?

PHP is server side, it does everything on the server and then displays it to the user as normal HTML, which if a search engine finds, it will show. If you're not using search engine friendly URLs though then it probably wouldn't find it since it's only picking up the index.php page.

Comment/Reply (w/o sign-up)

yordan

Feb 13 2008, 03:36 PM

By the way, pages needing passwords cannot be crawled, because the robots cannot guess your passwords.
So, pages you protected or scripts needing you to login before entering make the things not searchable by robots.

Comment/Reply (w/o sign-up)

rockarolla

Feb 14 2008, 03:08 AM

QUOTE(yordan @ Feb 13 2008, 03:36 PM)

Thats my point. The database table has an user/pass so it might not be accessible for a bot or it might be...I think they ``crawl'' a particular site by ``virtually opening'' it and then check the content...at least this is my experience...as I really need to know this I'm making some crawling around too to become an expert...

see how if I'll share it ...

here are the advises given by G:

http://www.google.com/support/webmasters/b...py?answer=40349

Comment/Reply (w/o sign-up)

yordan

Feb 14 2008, 08:56 AM

Hi, RockaRolla,
When I do crawling for fun, it simply opens the php scripts, so I get files named somehthing.php down to my PC, nothing more. For instance, in 4gallery photo galleries, I don't have the pictures.
However, I have the php scripts, so I can re-create the website. But of course I cannot connect to the database (for instance at asta it's your database so, even guessing the passwords I could not connect) so I cannot reach dynamic pages which are displayed using a particular user's rights and name and passwords.

Comment/Reply (w/o sign-up)

Quatrux

Feb 14 2008, 12:26 PM

Well, if your pages are dynamic, it would be very good for you to send the right headers to the bot/crawler and even to the browser, due to when apache or as I know any other http server sends a simple html file with the right content length and etc. but when a PHP file is generated/parsed apache can't really know the length of it, but it can be done with some php output buffering and sending the header with php, I think google is full of these kind of suggestions.

Comment/Reply (w/o sign-up)

Athlon1600

Aug 7 2008, 03:42 AM

that's what I thought too at first

don't worry about it, every extension is indexable to some point.
here is the useful tool:

http://www.seochat.com/seo-tools/spider-simulator/

enter your websites url and you will see your page exactly as google search engine spider would see.

Comment/Reply (w/o sign-up)

FirefoxRocks

Aug 7 2008, 05:01 AM

My thought on this is that pages needing POST form data (user logins, not database logins) are not accessible by search engines. This is because robots cannot guess the input required to access the content (passwords, email addresses, etc). If you have links to content outputted by PHP from a database, it should be searchable.

Comment/Reply (w/o sign-up)

toby

Aug 7 2008, 10:19 AM

You can tell google a user/pass, and sitemaps, but generally what it can't find a link to, it won't index.

Comment/Reply (w/o sign-up)

rnd-am

Feb 7 2009, 05:55 AM

Neither Google, nor any other search engine will not crawl ( and, hence, index) pages, that hadn't link on it on other, previously crawled, pages. So you can have entire universe in your DB, but if there is no link to those particular generated pages, then those pages are invisible for SE.

BTW ordinar forums are frequently set to settings which allow to browse pages even for not loggid in user, i.e. crawler, indexing robot.

Comment/Reply (w/o sign-up)