Creating One-Step Search Forms
Frequently Asked Questions
ORIGINAL VERSION

1. What is the purpose of this tool and who is it intended for?

This tool is intended for anyone who has a collection of data and wants to make it searchable from the web.  It enables you to easily generate the search form and search engine, create a database, and to add soundex codes to the database.

There are no fees or licensing involved.  It is all being offered free of charge.  You can use it and modify it in any way you like.  At the bottom of the search form that it generates is a reference to this tool.  You are free to remove that reference if you desire.

I am not offering a service to house your search form for you.  You'll need to have your own access to a server on which to place the generated search form, generated search engine, and your database.  There are many free servers out there for you to chose from, although they usually don't let you run search engines from such servers.  If you are willing to pay a little, you can get a server for only $5/mo that will let you run a search engine.  But if you really want to do it for free, this tool offers you a way to have the search engine run in the browser instead of the server (see the description of "Search Engine" in question 3), in which case you can run it on the free servers.
 

2. What is the difference between a search form and a search engine?

A search form is a form on which you enter values to be searched for.  It typically has a button that says "Search".  Pressing that button submits the values you entered to a search engine located out on a server.  That search engine goes through a database and finds the entries that satisfy the values you entered.  The search engine then generates the results as a web page that it displays on your screen.
 

3. How are the various entries under "Describe the Form" used?

The various entries that describe the form are:

Author: The author's name will appear below the heading line at the top of the resulting search form.  A copyright notice bearing the author's name will appear at the bottom of the form.

E-Mail Address: If an e-mail address is specified, the author's name at the top of the form becomes a link.  Clicking on that link starts an e-mail message addressed to the author.

Database Name: The name will appear in the title of the search form and also on the top line of the search form.  Specifically, it will say "Searching the xxx Database in One Step" where xxx is the Database Name you entered.  This name will also be used to find the database itself.  Specifically, the database must reside in a file named xxx.txt (for javascript search engines, the file name is xxx.js -- see question 10).

CGI Script is at: This gives the URL at which the search engine is located.  The search form needs this information in order to know where to submit the search request when the user presses the "Search" button.  This field is not needed if the Search Engine is written in javascript (see below).

Button text/value: Up to two buttons will appear at the top of the search form.  A typical use for at least one of the buttons is to link to a frequently-asked-questions page.  The button text is the wording that will appear on the button.  The button value is the URL that will be linked to when the button is pressed.

Soundex Type: Soundex is a method of finding matches that sound like the item being searched for although they might be spelled differently.  There are several different soundex encodings available.  American Soundex is typically used by the National Archives for census records.  Daitch Mokotoff soundex is useful when dealing with eastern European names.

Search Engine: There are various languages that the search engine can be written in.  The search engine typically runs on the server.  Typical languages supported by servers are "Perl" and "PHP".  Not all servers support these languages so you'll need to check with your system administrator to find out which language is supported.  If neither are supported, you can run the search engine entirely in the browser.  In that case select javascript ("js" or "js-2") as the language.  See question 15 for the difference between js and js-2.


4. If my server supports both Perl and PHP, which one should I use?

Use PHP for the following reasons.

Perl on some servers has a nasty habbit of requiring that the line separators be a single line feed (LF) character without any carriage return (CR) character.  And, unfortunately, most windows-based text editors create a CR-LF pair at the end of each line.  So you'll need to find some means of removing the CRs from your Perl script.

Other problems with Perl scripts are that they usually have to be put in a special directory on the server, the first line of the file usually needs to be modified to point to the perl interpreter on your server, and they usually need to have their access rights modified so that they are executable.  If you don't know how to do any of this, you certainly don't want to use a Perl search engine.  See question 7 for more details.
 

5. How do I define the searchable fields that I want on my search form?

There is a dropdown list that shows all the fields.  Initially a single field is defined.  You can create additional fields by clicking on the "Create New Field" button.

For each field you can specify a set of properties.  You do so by selecting the field in the dropdown list, and then entering the properties of that field into the appropriate boxes.  The properties of the selected field are:

Field Name: This is the text that will precede the box for the field on the search form.

Database Column: The database consists of one line for each record in the database.  Each line consists of columns containing the fields of that record.  This property tells which column contains the selected field.  The column numbering starts with 1.

Screen Column: Once a search is completed, the records in the database that match the search criteria are displayed on the screen.  Each line of the display consists of columns corresponds to the fields of one record.  This property specifies in which column the selected field will be placed.   The column numbering starts with 1.  If you leave this property blank, the corresponding field will not be displayed.  See question 15 for a discussion on how the user can see such fields.

Search By: Specifies the type of search that can be done on the selected field.  The first five choices (is exactly, sounds like, starts with, contains, ends with) are used with text string fields such as "name", whereas the last choice (is between) is used with numeric fields such as "age".

Use Indexing on this Field: Permits faster searches by allowing you to search through a single letter rather than the whole alphabet.  Only one field can be selected for indexing.  See question 17 for more details about indexing.


6. How do I create my search form?

Once you've described your form and defined your fields, you are ready to create the search form.  You do this by pressing the button that says "Search Form".  A page will appear with the html code for the form.  Select the code and copy/paste it into a text editor.  The details steps for accomplishing this are:

Select the html text by using the edit menu or by typing ctrl-a

Copy the selected text by using the edit menu or by typing ctrl-c

Open a text editor such as notepad

Paste the selected text into the word processor by using the edit menu or by typing ctrl-v

Save the file making sure that the name ends in .htm or .html

At this point you can examine the search form by opening the file in your browser.  If you are familiar with html code, you can do some tweaking of the search from at this time by hand-modifying the html code.  Once you are satisfied with it, you can upload it to your server to make it accessible to others.
 

7. How do I create my search engine?

The details for doing this are exactly the same as for creating the search form (see question 6), except now you press the botton that says "Search Engine".  The search engine cannot be tested out on the client and must be uploaded to a server.

If your search engine is written in Perl, you may have to perform some of the following steps as well.  These steps are not needed for PHP search engines.

- you might need to remove the carriage return characters from the text file of your search engine (see question 4).  One way to do this to use an editor that does not insert such characters.  Another is to use a utility that strips out the carriage returns.  Another is to use an upload facility (such as a file manager provided by your server) which removes the carriage returns as it uploads the file.  You might need to consult your system administrator for advice on how to do that.

- you'll need to modify the first line of the file to point to the location of the Perl interpreter.  Check with your system administrator to find out where that is.

- you might have to put your search engine into a special directory such as cgi-bin.  Check with your system administrator about this.

- files on a server have attributes indicating whether or not they are executable.  You'll probably need to have the executable attribute set.  The means of accomplishing this will depend on your server, so you should consult your system administrator to find out how to do it.
 

8. What format must my database be in?

The database consists of one line for each record in the database.  Each line consists of tab-separated columns containing the fields of that record.  The determination of which column corresponds to which field is specified by the "Database Column" (see question 5).

Note that adjacent fields are separated by a single tab character.  The tabs are not used to create visual alignment.  You must not add additional tabs to make the alignment nicer because such extra tabs will alter the meaning of the subsequent fields on the line.
 

9. My database is in a spreadsheet.  How do I get it to be in the tab-separated format that you require?

Your spreadsheet program has a "save as" menu item under the file menu.  You can specify a type for the file being saved.  Specify the type to be "tab-delimited text file".
 

10. What does the "Database" button do and how is it used?

If you are generating a javascript-based search engine, the data has to be reformated slightly in order to accomodate the javascript syntax.  Also, in order to do a sounds-like search, soundex codes must be inserted into the database.  If you are not using a javascript search engine and you do not intend to do any sounds-like searches, you can skip this step.

You can reformat the data for javascript and/or add the soundex codes to the database as follows:

Press the "Database" button

A page will appear with a box into which you can insert your database.  Copy-and-paste your database into this box.

Click on the button that says "Process".

A new database will appear in the bottom box.  This database has all the fields that were in the original database, as well as additional fields that contain the soundex codes.

Copy and paste this resulting database into a file.  (Note that on the Netscape 4.x browser, you cannot use ctrl-a to select this data for copying.  Instead you must right-click the mouse and then do a "select all".)

Name the database xxx.txt where xxx is the name you selected for your database.  However if you are generating a javascript search engine, then you must name this database xxx.js (if you are using indexing, the file is xxx.html -- see question 17 for details).

Upload the xxx.txt or xxx.js file to the server. Place the file in the same directory as the search engine.  If you have selected js (but not js-2) as your search-engine language, then the search engine is embedded in the search form and you need to place the xxx.js file in the same directory as the search form.


11. Is that all there is to it?  Can you give an example?

Yes, that's it.  At this point you can bring up the search form with your browser, fill in the appropriate fields, and click on the search button.  The search request will be sent to your search engine which in turn will search through your database.  The results of that search will then appear on your screen.

Here's an example.  Suppose you have a database consisting of last-name, first-name, and age in that order.  However you would like your results to appear in the order first-name, last-name, age.

Suppose the raw database is in the file named rawdata.txt.  It looks as follows:

Smith<tab>John<tab>21
Doe<tab>Jane<tab>35
Jones<tab>Henry<tab>18
The string "<tab>" above designates the tab character.  The actual file must contain the tab character and not the string "<tab>".  Also the above lines are indented to make them easy to see in this text.  The actual file must not be indented.

You would now make the following entries in the "Describe the Form" section on the "Creating One-Step Search Tools" form.

Author: Your name
E-Mail Address: Your e-mail address if you would like people to be able to contact you
Database Name: Family
CGI script is at: http://yourserver.com/cgi-bin/family.cgi
Button 1 Text: Frequently Asked Questions
Button 1 Value: faq.html
Button 2 Text: Useful Webpages
Button 2 Value: http://stevemorse.org/index.html
Soundex Type: American
Search Engine: Perl
Next you would define your fields as follows:
Edit the first field so that it contains the following properties:
Field Name: Last Name
Database Column: 1
Screen Column: 2
Search By: is exactly, sounds like, starts with, contains, ends with
Press "Create New Field" and edit the new field so that it contains the following properties:
Field Name: First Name
Database Column: 2
Screen Column: 1
Search By: is exactly, starts with, contains, ends with
Press "Create New Field" again and edit the new field so that it contains the following properties:
Field Name: Age
Database Column: 3
Screen Column: 3
Search By: is between
Now you are ready to create your search form and search engine.  Press the "Search Form" button.  The html code for it will appear.  Select all of it (ctrl-a) and copy it to the clipboard (ctrl-c).  Use notepad to open a new file named search.htm.  Paste (ctrl-v) the contents of the clipboard into that file and save it.  Upload it to a publicly-accessible place on your server.

Similarly create the search engine by pressing the "Search Engine" button.  Save it in the file named family.cgi.  Upload that file to the cgi-bin directory or other appropriate location on your server.  You might need to remove the CR characters from the file if this is a Perl script.

Now you can append the soundex codes to the database.  Press the "Database" button.  A screen with two large boxes will appear.  Copy your entire rawdata.txt file to the clipboard (ctrl-a, ctrl-c) and then paste it (ctrl-v) into the upper box.  Press the "Process" button.  The database will now appear in the lower box with the soundex code for the lastname field added.  Note that only the lastname field gets a soundex code because it was the only field that you designated as being searchable by "sounds like".  Now copy this resulting database to the clipboard (ctrl-a, ctrl-c) and past it (ctrl-v) into a new file named Family.txt.  Upload this new file to your server in the same directory that you uploaded your search engine.

Finally you'll want a frequently-asked-questions page so you won't have to keep responding to the same questions over and over again.  Use whatever html editor you have available to create such a file, name it faq.htm, and upload it to the server in the same directory that you uploaded the search form.
 

12. What does the "Test Drive" button do?

The "Test Drive" button is similar to the "Search Form" button in that both create the html code of the search form.  However the "Search Form" button displays the generated html code whereas the "Test Drive" button executes that code.  In other words, when you press the "Search Form" button you will see the generated html code, and when you press the "Test Drive" button you will see the form that the generated html code produces.

You can use the test drive to quickly test out your search form since it saves you the step of having to copy your html code to a file and then bringing up that file in your browser.  Once you are satisfied with the generated search form and are convinced that it does the search properly, you can then save the search form to a file so that you can bring up that file in your browser in the future.

The test drive let's you both see you test form and perform an actual search using the test form.  But in order to do the search, you will first have to create the search engine and database, and upload both of them to your server.

There is a problem if you have selected js as your search-engine language.  The test drive will allow you to view the search form, but you will not be able to see your database file.  So you will not be able to do an actual search.  Even though you can't do a search, you can at least see what the resulting search form looks like and determine if it meets your needs.  This limitation does not apply to the other search-engine languages (Perl, PHP, or js-2).

And there is another limitation of the test drive.  If you are using a Netscape 4.x browser, you can not do more than one search during each test drive.  You will get a warning message if you try.
 

13. I have a less-than symbol in one of the entries in my database.  Why doesn't it display properly on my results page?

The less-than symbol has special meaning as an html tag.  For example, suppose you want to put an e-mail link into one of the fields of your database.  In that case you would enter the value in your database as:

<a href="mailto:john@doe.com">John Doe</a>

If you would like to enter a real less-than symbol into one of your database fields, you need to enter it as a pair of less-thans.  For example, to enter x<y into a field, you would put it in as x<<y.

See also question 14
 

14. I have many ampersand symbols in my database.  Some display fine but others do not.  How can I get them all to display?

Unfortunately ampersand (&) has a special meaning in html code.  Specifically it denotes certain characters.  For example "&lt;" denotes the less-than character (<).  So if you happen to be unfortunate enought to have an ampersand in your database and it is followed by "lt;", it will display as "<" instead of the string "&lt;".

For the most part that shouldn't present any problems because it is unlikely that you would have a valid character designation following the ampersand.  But if you do, you will not get the display that you expected.  To avoid this problem, you can double up your ampersands.  Two ampersands together will always display as a single ampersand and will never look like the beginning of a character designation.

See also question 13.
 

15 What is the difference between "js" and "js-2" as the search-engine language?

See question 3 for a description of search-engine language.

Both js and js-2 will result in a search engine written in javascript and running in the browser.  The difference is that js will result in the search engine being embedded in the search form itself.  In that case the search engine will not be in a separate file.  That is why you do not need to specify the location of the search engine ("CGI Script is at") when using js as the search-engine language.

So it would seem that js is preferred since there is only one file involved, and there would never be a reason to use js-2.  However there were some unorthodox tricks used to allow the search engine to run inside the search form.  These tricks prohibit the use of a link on the results page to display the next or previous set of matches.  For this reason, js-2 is actually more desirable to use even though it means that you will have two separate files -- one for the search form and the other for the search engine.
 

16. What do I do if my data has more fields than can be displayed neatly on one line?

The results of a search are displayed to the user in tabular form, where each line of the table corresponds to a match that was found and each column on the line are the fields of that match.  For example, if one of the matches corresponds to a person named John Smith age 25, one line of the table would contain the fields "John", "Smith", and "25".

You can specify which fields are to be displayed in the table and in what order.  You do this with the "database column" property (see question 5).  If you leave that property blank, the corresponding field will not be displayed.  This is useful when you have more fields than will fit on a single-line display.

If there are any non-displayed fields, a link entitled "details" will appear at the end of each line displayed.  If the user clicks on one of those links, a separate display corresponding to the particular match will appear.  That display will list all fields of that match.  This allows the user to see the values of the fields that are not displayed in the results table.
 

17. What is indexing and how do I use it?

Indexing is a means of speeding up the searches by looking under a specific letter rather than the whole alphabet.  For example, if the lastname of the person being searched for starts with R, it would be quicker to search only through the R's rather than search through the entire database.

Only one field can be selected for indexing.  You make this selection by checking off the "Use indexing on this field" box under the field's properties (see question 3).  You will also need to subdivide your data into 27 files -- one for each letter of the alphabet and one catch-all for any values that do not start with a letter.  You can still use the "Database" button for creating your database (see question 10), but you will have to create each of the 27 files separately.

If you are using a Perl or PHP search engine, name your database files xxx_a.txt, xxx_b.txt, etc where xxx is the name you assigned to your database.  Name the catch-all file xxx_0.txt.

If you are using a javascript search engine, name your files xxx_a.html, xxx_b.html, etc., and name the catch-all file xxx_0.html.  Note that this differs from the non-indexed case in which the database file had a .js suffix instead.
 

-- Steve Morse