Category: Tech

ICANN is issuing new top level domains faster than I can create new spam filters to stop the trash coming from the likes of .top, .download, ,mobi, .date, .xyz, .click, .rocks, .wang ….

Supposedly, there really are legitimate web sites using these TLDs, but I personally have yet to actually SEE any of them. The ONLY reason I even know most of these exist is the sudden appearance of a ton of spam from each one of them.

I administer the filters for a number of email addresses, and some of the owners have not been as careful as others and their email address has gotten on some spammer’s list. And once you are on one, you will shortly be on scores of them.

I have yet to see a legitimate email from ANY of the new TLDs.

ICANN thought all these new TLDs would be a great idea, a real boon to the internet. Well, they certainly have been a boon to ICANN and a bunch of shady businesses which are making mucho dinero off their creations! For the rest of us, they are more of a… I’ll be polite and say nuisance.

It’s time for ICANN to pull in the reins, maybe even start phasing out the use of some of the new TLDs. For those legitimate websites (if there really are any) with one of these fad TLDs, my advise would be to give it up get a real TLD. Then I might actually visit you.

Find the new Sphider 1.5.0 on our Downloads page.

UPDATE: 1 December 2015,   18:25 UTC. If you downloaded 1.5.0 before this time, the auto-suggest may not work if you installed Sphider to any directory NOT named /sphider. The current posting DOES work.

If you are affected, you do not need to re-download. Simply find “autocomplete.js” in the js_suggest directory,and edit line 7 from:
$.get( “/sphider/js_suggest/suggest.php”, { keyword: keyword } )
to
$.get( “js_suggest/suggest.php”, { keyword: keyword } )

85	CVE-2014-5194	94	1		2014-08-07	2014-08-07	6.5
Static code injection vulnerability in admin/admin.php in Sphider 1.3.6 allows remote authenticated users to inject arbitrary PHP code into settings/conf.php via the _word_upper_bound parameter.

86	CVE-2014-5193	79	1	XSS	2014-08-07	2014-08-22	4.3
Cross-site scripting (XSS) vulnerability in admin/admin.php in Sphider 1.3.6 allows remote attackers to inject arbitrary web script or HTML via the category parameter. NOTE: the url parameter vector is already covered by CVE-2014-5082.

91	CVE-2014-5082	89	1	Exec Code Sql	2014-08-06	2015-11-04	7.5
Multiple SQL injection vulnerabilities in admin/admin.php in Sphider 1.3.6 and earlier, Sphider Pro, and Sphider-plus allow remote attackers to execute arbitrary SQL commands via the (1) site_id or (2) url parameter.

And it all started because the deprecated code was making Sphider useless. The deeper we dug…

Ain’t that the way life works? You start out fixing one problem and find two more to take its place.

Have we fixed ALL the problems? Probably not, but it is a darn good start!

Version 1.5.0 is just days away from public release. Testing is continuing, but going very well. The NEW Sphider User’s Guide is essentially complete and being reviewed. We could say it is ready to go now, but rather put it through its paces a couple more times before publishing it on Tuesday, December 1, 2015.

So what’s so special about Sphider 1.5.0? For starters, it is fully up to date. PHP 5.6 loves it. So does MySQL 5.6. And all the html is HTML5, which is very strict in structure. Because Sphider is written in PHP, if you do a “view source” on a web page it ain’t gonna be pretty. But it is correct.

Having read the change reports for MySQL 5.7, which was just recently released, no changes appear to be needed. And, although still in Release Candidate stage, early examination of the coming PHP 7 does not reveal any issues. Sphider 1.5.0 is ready!

Security has also been a concern. Sphider 1.3.6 is ripe with opportunity for SQL injection attacks. Simply getting rid of the deprecated MySQL extension and replacing it with the MySQLi extension did nothing for security. Moving to prepared statements DOES virtually eliminate SQL injection attacks. With prepared statements, bound variables are kept separate and never parsed as a part of a generic SQL statement.

Wherever GET, POST, or REQUEST data is used, it is escaped, matched, and any otherwise reduced to safe data.

One critical Sphider page was once COMPLETELY rewritten, using unescaped GET data everytime the settings were changed. No more. This page (which you never actually SEE), is now static in structure and completed on call from the database. Of course, changing the configuration means updating the database, which in turn uses GET data. The thing is, now 1) the GET data is parsed and escaped, and 2) is written to the database using the prepared statement process. This critical page can no longer be hijacked and used as a weapon against you.

Originally, there was some PHP code written into some .html pages. If you looked at the page in a browser and went to “view source”, anyone could view snippets of actual PHP code. No more.

We also found that, if you dug deep enough into the spider functions, our earlier efforts to improve Sphider broke a couple things. We corrected those. We ALSO found there were things in 1.3.6 that were SUPPOSED to work, but didn’t. We corrected those, too.

So, are we claiming Sphider 1.5.0 to be bug free, the perfect Sphider? No, we aren’t that vain. But, for today’s environment, Sphider 1.5.0 is a good fix for the dying Sphider 1.3.6. And you won’t have to pay money for Sphider-plus or Sphider-pro and get functionality you don’t need.

We feel the Sphider User’s Guide is l-o-n-g overdo. Wouldn’t it be nice to really know what all the setting do, what happens on each of the admin pages, and what kind of searches you can do? It’s always nice to have a road map.

(Did we mention that spelling suggestions now works much more reliably? I supposed it USED to work before, but in today’s world, it was a no-show on most browsers we tried.)

Watch our Downloads page. Sphider 1.5.0 should be making its appearance there this coming Tuesday.

A long time ago, WorldSpaceFlight had nothing but html pages. Indexing the site was a pretty easy task using a simple perl script, and a user search was pretty easy.

Time passed, and the first php pages were introduced, pages which could not be indexed by the perl script. What started as a minor problem with a few pages grew into a MAJOR problem with nearly ALL the pages as we progressed towards a complete (almost) use of php pages.

Then enter Sphider. This is a tool from www.sphider.eu which allowed php pages to be parsed an indexed. Things were great. But time passed, the state of PHP and MySQL advanced, but alas, Sphider did not. With only one security update, Sphider has remained the same for the last 6 years. Security problems, deprecated code, things that just plain no longer worked…

Sphider became useless. Looking for a suitable replacement proved fruitless.

So, we did what any practical person would. If the tool was broken, FIX IT! And we did, but just for ourselves. We had a Sphider that worked, but it was still a huge security risk, highly vulnerable to SQL injection or remote code execution attacks. We protected ourselves the best we could, but finally decided that just wasn’t enough!

The result was Sphider 1.4.1. Sphider was considerably hardened. But that wasn’t enough for us. We wanted more, and since we had already dug into Sphider’s code, we did more. We added a wildcard search and some new templates. The default “standard” was, yawn, boring! And the “dark” template was, well, dark! And since the PHP was up to date, we made all the HTML up to date, HTML5. We also made it possible to index pdf files in a Windows environment, and not just in Linux. We threw in a few bug fixes while we were at it. Thus was born Sphider 1.4.2.

Not satisfied with what we had, we have been working on yet ANOTHER update. The suggest function, which USED to work way back when, had become rather finicky, working for some browsers, not for other. So we reworked the whole”suggest” feature. Then we converted all queries to use prepared statements. This virtually eliminates SQL injection attacks. More bug fixes, a couple enhanced features here and there. The code has been cleaned up considerably. This will be Sphider 1.5.0. It is a MAJOR change from 1.4.2. We are still testing, as we made SO MANY changes we want to be sure nothing got broken in the process. When it is ready, there will be a User’s Guide to go with it, something which has been lacking. Searching the forums can only go so far.