Archive for the ‘Announcements’ Category

Search Engine Submission API Is Up Again

Saturday, October 9th, 2010

In case you have been trying to add your URL though any of the automated services that use our API, it might have failed.

You see, we had a problem with our web hosting provider. And that prevented most confirmation emails from being sent out.

I get the problem fixed now, by moving the confirmation emails to another server. Still if you submitted though FreeWebSubmission.com or through another automated service between 4th and 8th October you URL is most likely not in our index.

To those of you that were hit by this I just want to say I’m sorry for the hassle.

To get your URL indexed, just go to our add url page and enter it manually. That’s all you need to do.

Happy Searching,

Simon
CEO and Founder
Secret Search Engine Labs

Tags: , , ,
Posted in Announcements, Bugs | 4 Comments »

Search Engine Query, Parsing Improved

Saturday, February 27th, 2010

Improved search engine query parser and with that more accurate search results, that’s what you can expect from last days updates.

I just had the privilege of spending a day programming instead of doing marketing. I used the time to improve the search engine query parser to find long keyphrases in a query and to remove redundant shorter phrases that are really just a sub phrase of one of the long phrases.

The problem with the old search engine query code was that it would sometimes omit common words even though they were essential to the search engine query. It could omit “best” in the query best hosting providers, and that changes the meaning of the query.

It also split search engine queries into far too many sub-queries.

Like in this search for Hotels in New York City, it’s split the old query parser into “hotels”, “in”, “new”, “York”, “City”, “Hotels in”, “New York”, “in New York”, and “New York City” which is a lot of redundant queries and may end up giving New York too much weight at the cost of Hotels

With the new search enging query parser in place the query for “Hotels in New York City” will omit “in”, “city”, “New York” and even “New York City” as they are sub phrases of the longer “in New York City”

Not that the new query parser is perfect, it still fails to search for “Location: New York City”, Hotels in Manhattan or an address in New York City.

The solution to this of course is the planned synonym logic that will make a search for “in New York City” also query alternatives like “in NYC”, “of New York City” “in Manhattan” etc.

It will be some time though before I have synonyms running as a fair amount of processing power is needed to determine synonyms form the main index. I currently have one Linode working on it and I expect to be able to test the algorithm in a month or two.

Update: I found an interesting article from SEO by the Sea describing a Google patent on Search Engine Query Statistics. The huge volume of search queries that Google has in it’s logs lets the draw a lot of useful conclusions on word relationships and user behavior. I would love to have that much data in my query log :)

Please leave you comment and tell me what you think about the new search engine query parser and please also leave a comment if you notice any interesting phenomenons with the new search query analyzer :)

Simon
Secret Search Engine Labs

Tags: , , , ,
Posted in Announcements, Development, Tech | 2 Comments »

Big Server Upgrade Lets Us Increase Index Size and Paves Way for New Algorithms and Public API

Friday, January 22nd, 2010

Yesterday I completed the promised server upgrades, upgrading the main server from a Linode 720 to a Linode 1080 and adding a new Linode 360 server to host the API (I’ll tell you more about that later) and for doing special data processing that is outside the normal indexing cycle.

So what does this mean for you?

The main thing you’ll notice is that the index will get bigger. I can’t tell you how much though as this is impossible to predict, but I really hope we will be able to reach 300,000 pages with this setup.

There’s two things that will allow the undex to grow:

1. With more RAM, 1080MB instead of 720MB the updating of the index database will be quicket letting the robot index more pages in a month.

2. As there is less VPS nodes sharing the same server there should be more disk and CPU cycles available to Us. This is not cut in stone though as it depends a lot on what usage profile the other nodes have and I don’t know that yet.

The index will grow, great. What about that other server then?

The other server, the Linode 360, will be used to host the new API/Feed that I will announce soon. I will make a feed of the search results available for free letting you make your own search engine, use it as additional content for your directory or anywhere that you want to give your visitors some relevant websites to visit. But more about this later.

The other mission for the new server will be to do calculations of data sets that will support the main indexing.

Mission one will be to make a related keywords database to allow us to find sites about “New York City” when someone writes “NYC” or “Win XP ” when someone writes “Windows XP”.

I know I promised you an index of 300,000 pages the last time we upgraded servers but things changed and I implemented a few new algorithms that improv quality but slows down the indexing process.

The main slowdown this year has been the addition of the search cache, a separate database arranged to make search queries ultra fast. You can now expect to get your results in under a second, sometimes in two, while before the cache searches was anywhere between 4 and 40 seconds as they had to dig through the main index.

Now lets just wait and see how much the index grows.

Simon Byholm
CEO and founder,
Secret Search Engine Labs

P.S. If you order a Linode through the links in this post Linod will give us $20 in free server time.

Tags: , , , ,
Posted in Announcements, Tech | No Comments »

PRESS RELEASE! We Are Joining The Independent Search Engine And Directory Network (ISEDN.org)

Wednesday, January 20th, 2010

I have the pleasure to announce that late yesterday the final steps was taken to integrate the advertising feeds from ISEDN.org into the search results pages. (It’s the green text ads to the far right)

On December 26th (between eating leftovers from Christmas) I signed the partner agreement with Jayde Online Inc to join the Independent Search Engine and Directory Network (ISEDN.org) also known as the ExactSeek featured listing program.

The feed is now live (for an example see the search for webmaster) and you can get a featured listing for your website targeting a specific keyword. Your ad will show up on Secret Search Engine Labs and on over 375 other search engines in the ISEDN network.  See the advertising page for details.

Joinig the ISEDN will allow us to increase revenue from the search result pages which in turn mean we will be able to get bigger and better servers to increase the index size and improve the search results.

The press release about this just went live on PRWeb and I’d like to say welcome to all new visitors, please try the search results and tell me what you think by leaving a comment below.

Simon
CEO and founder,
SecretSearchEngineLabs.com

Tags: , , ,
Posted in Announcements | 1 Comment »

Beta Testing Is Open, Did You See The Press Release Last Week?

Tuesday, January 12th, 2010

I forgot to blog about it but in case you didn’t notice we started the official beta testing period last week and we even had a press release distributed through PRWeb.

Thanks to the press release we had about 400 extra visitors on last Thursday and Friday,  most spending more than the average number of click on site.

The number if visitors isn’t exactly earth shattering but nevertheless it was great to have you all here and I hope many of you will be back for more searches.

The Beta testing period is planned to run all year and end in a public launch in January 2011 with the target of having over a million relevant pages indexed by then.

I also want to take the opportunity to apologize if you have stumbled upon some less than relevant results. We had a bit of a Cuil experience when in anticipation of a stampade of visitors from the press release we added a new cache database to speed up searches.

Yes the searches are now about 30 times faster with most searches ready in less than half a second.

The downside is it will take a whole month before the cache is fully populated with data and in the meantime the search results are based only on a subset of the whole index. For some search terms this means junk and garbage from the bottom of the barrel will show up.

Please enjoy the new super quick search results and check out the ping tool in the webmaster tools section.

Tags: , , , , , , ,
Posted in Announcements, Bugs, Tech | No Comments »

Thanks to Server Upgrade We Can Now Index Up To 300,000 pages

Wednesday, December 16th, 2009

A couple of days ago we upgraded our VPS server from a Linode 360 to a Linode 720 which means we now have doubled our indexing capacity and you should see a steady growth in index site during the coming month.

In reality even though we doubled the server resources the indexing speed has, according to the stats, increased from about 6,000 pages/day to around 10,000 pages per day. I believe the reason we don’t see a doubling of indexing speed is the fact that we still run the indexer on a single thread which means we are spending too much time waiting for network transfer and disk access,  not utilizing the CPU even if we could.

Which just makes it that much more important to get the multi-threaded indexer implemented… just need to get some marketing done first.

The migration to the new servers went really smoothly. With the system that Linode uses you just select the upgrade you want from the online interface and once you have clicked the “Yes” button they instantly and automatically shut down the site and copy everything to the new bigger system.  It all took 12 minutes of downtime plus a couple of minutes to reconfigure mysql for the larger memory size.

If you’re looking to get a VPS server that is easy to upgrade Linode is a great alternative and you can support this search engine by signing up for a Linode using this link.

Linode will support us with $20 for every one of you that stays a Linode customer for more than 90 days.

Please tell me what you think about the quality of the search results by leaving a comment below!

Simon Byholm
CEO and founder,
Secret Search Engine Labs

Tags: , , , , ,
Posted in Announcements, Support Us | 2 Comments »

Secret Search Engine Labs Now Have a Blog for News and Announcements

Wednesday, December 9th, 2009

Today I finalized the setup of a WordPress blog for Secret Search Engine Labs.

The new search engine blog will be used to send out news and announcement related to the work I do at Secret Search Engine Labs as well as for discussing new ideas for ranking and filtering data.

Feel free to add your ideas, criticism and praise by commenting on the blog posts, I love feedback. The search engine is meant to search you in the best possible way and that will be possible when you tell me what you expect of the worlds most useful search engine.

Have a great day,

Simon Byholm
CEO and founder,
Secret Search Engine Labs

Posted in Announcements | 1 Comment »