Tag: Sphider

The backup and restore utilities have been reworked to use MySQL directly. This provides higher dependability than depending on PHP.  Also, a limited ability to resume a re-index process which has been interrupted has been introduced. The process to determine page character set has been enhanced. Language file conversion to Unicode has been completed. Obsolete versions of code have been removed and general code cleanup done. Further safeguards against indexing of illegal characters has been implemented. SphiderLite has had more remnants of the full version removed.

A critical flaw was discovered in SphiderLite which affected the indexing of URLs. This has been corrected in 1.3.1.

All users of previous versions of SphiderLite are urged to STOP USING IMMEDIATELY and upgrade to version 1.3.1. It does seem that initial indexing with no special circumstances was successful in prior versions, re-indexing was adversely affected. If Sphider was allowed to leave the domain, there could also have been adverse effects.

The cause of the problem? SphiderLite is a scaled down version of the full featured Sphider. When the full version was scaled down, some function parameters unique to the the full version were inadvertently retained, throwing off the parameter sequence in the Lite version.

SphiderLite 1.3.1, in addition to fixing this critical flaw, has also improved the method of determining a page’s character set, improved filtering of emojis, and improved filtering of unwanted characters in the indexing of keywords.

Sphider 3.6.0-MB is UNAFFECTED by the flaw discovered in SphiderLite. It will, however, soon be updated with the same additional improvements in this Lite version.

A potential runaway regular expression resulting in missing titles has been corrected. Crawl performance has been improved by fixing a bug that caused Sphider to try to crawl pages returning codes like 301, 401, 403, and 404. The absence of a robots.txt file on sites being crawled was generating warning errors, and this has been corrected. More potential PHP 8 errors have been averted. More obsolete code has been removed. The MB version now reports when a feed becomes invalid.

A change to how Sphider does searches was very recently implemented. It was found, however, that everything worked fine, PROVIDED all searches yielded results! If a search should not have any valid results, a message saying “The search for [search] yielded no documents” is supposed to be displayed. Instead, such a search actually presented the results from the last successful search! NOT GOOD!

It was found that session variables were not being cleared in the event a search yielded no results. That has been corrected.

Sphider is multibyte capable. It has been since Sphider 3.0.0-MB. Sphider 3.0.0 only worked on installations which had the PHP mbstring module installed. Sphider 3.1.0-MB and later works on all installations, and if the mbstring module was missing, it was emulated. Obviously, if the module was present, emulation was not needed. Well, one emulated function neglected to check for mbstring and ALWAYS emulated the function. This version corrects that. The result is that if an installation has mbstring installed, searches will run faster than before.

Versions prior to Sphider 3.5.1 and SphiderLite 1.2.1 will work, but just not nearly as efficiently.

The text search feature has been updated to provide more efficiency and quicker response. Previously, a search was repeated for every page of results. This has been changed so that a particular search is performed only once. Then the appropriate subset of results is displayed for each page. This does not improve searches with only 1 page of results, but each page thereafter will see an improvement.

This release fixes problems with robots.txt files, removes obsolete database functions, and removes code deprecated in PHP 7.4 and removed in PHP 8. Barring surprises, both versions of Sphider should be PHP 8 compatible.

Maintenance releases for the PDO version of Sphider have ended and the PDO version has been removed from the general download library.

Sphider 3.4.4-MB and SphiderLite 1.1.4 have been released. These releases fix a critical bug causing the database truncate table function to fail. The PDO version was unaffected.

The Sphider 3.4.3-MB and SphiderLite 1.1.3 releases fix a number of issues which arose under PHP 7.3 and PHP 7.4.  It was found that as of PHP 7.3, image indexing in Sphider MB releases had broken. This has been resolved. A couple of cleanup items were also done with each.

Sphider 2.4.3-PDO is a one time maintenance release to keep the PDO version running under PHP 7.3 and PHP 7.4. It is not intended to resume development on the PDO version at this time.

Both Sphider and SphiderLite have been updated to correct code which became deprecated with the release of PHP 7.4. Functionality has remained unchanged.