A couple of days ago we upgraded our VPS server from a Linode 360 to a Linode 720 which means we now have doubled our indexing capacity and you should see a steady growth in index site during the coming month.
In reality even though we doubled the server resources the indexing speed has, according to the stats, increased from about 6,000 pages/day to around 10,000 pages per day. I believe the reason we don’t see a doubling of indexing speed is the fact that we still run the indexer on a single thread which means we are spending too much time waiting for network transfer and disk access, not utilizing the CPU even if we could.
Which just makes it that much more important to get the multi-threaded indexer implemented… just need to get some marketing done first.
The migration to the new servers went really smoothly. With the system that Linode uses you just select the upgrade you want from the online interface and once you have clicked the “Yes” button they instantly and automatically shut down the site and copy everything to the new bigger system. It all took 12 minutes of downtime plus a couple of minutes to reconfigure mysql for the larger memory size.
If you’re looking to get a VPS server that is easy to upgrade Linode is a great alternative and you can support this search engine by signing up for a Linode using this link.
Linode will support us with $20 for every one of you that stays a Linode customer for more than 90 days.
Please tell me what you think about the quality of the search results by leaving a comment below!
Simon Byholm
CEO and founder,
Secret Search Engine Labs
Tags: index size, linode, search engine, support this search engine, upgrade, vps server
UPDATE: We did a software upgrade that increased query speeds from an average of 15 seconds/query to less than a second per query.
Too bad this uses some resources and the indexing capacity is back at 200,000 pages.
The user experience while searching is greatly improved though and I promise we will do another server update shortly.
Simon
My friend and I were recently discussing how we as a society are so hooked onto electronics. Reading this post makes me think back to that debate we had, and just how inseparable from electronics we have all become.
I don’t mean this in a bad way, of course! Societal concerns aside… I just hope that as the price of memory drops, the possibility of downloading our brains onto a digital medium becomes a true reality. It’s a fantasy that I daydream about every once in a while.