Main Content

Crossrefering documents with uniqueness and inconsistency issues - PHP proof of concept demo

Archive - Originally posted on "The Horse's Mouth" - 2009-05-10 12:11:47 - Graham Ellis

The cross-referencing of documents - where one document contains a human readable reference to another - is very common. And so is the requirement to convert those cross-references into a appropriate links.

The Example

Let's say that I have chunks of data of up to 160k, relating to up to 4000 images, as output from some sort of search. Some of data lines have reference codes ("see also" stuff) attached to them. And I want to come up with a table of the "see also"s There can be several attached to a single line of data, and the formatting is inconsistent because it's manually updated. And a "see also" reference can occur a number of times. Sound like s**t loads of potential problems?

Here is some sample data:

train.jpg A train pulls in to Melksham Station
eiders.jpg Eider ducks (BIRD 15/025/35)
diffdir.jpg A view full of terns (BIRD 15/44.3/19)
pufffly.jpg Puffin in flight (BIRD 33/23/33)
puffnests.jpg Puffins nesting (BIRD 33/23/ 33)
gullery.jpg Gulls nesting at Vik (BIRD 44/32/123)
yern2.jpg A tern searches for food ( BIRD 15/44/19)
DSC08354a.jpg Santa at Melksham Station (PERS 43/22/12)
DSC08379.jpg Father Christmas at Melksham (PERS 43/022/12)
leveepics.jpg Lisa snaps up California (PERS 36/26/36)
avbr6.jpg Ascending a lower lock
newb_2.jpg Gypsy and Graham (ANIM 33/23/77) (PERS 12/73/17)
newb_3.jpg Gypsy (ANIM 33/23/77)
avbr4.jpg Lower down the locks at Caen Hill
avbr1.jpg Avebury Church

And I want to get a table of all the cross-referenced subjects (e.g. PERS 43/22/12 - Santa Claus) and all the data lines that each relates to.

The Solution

I have written a "proof of concept" program in PHP which analyses my data in the format above and produces a table of each reference, with a link to the image that it refers to.

The principle of how it works is as follows ...

The PHP program reads the data source record by record, and finds any references within each record. Each reference is canonicalised (reduced to a standard form for consistency) and stored into an associative array of results, so that we can spot non-unique references instantly and deal with them.

Once we have parsed the whole data source, all we need to do is to sort the array of references, and loop through the output to display it.

It sounds and IS simple ... but the first time you do something like this you have to be very careful to use the correct, efficient technique to deal with the uniqueness and irregularity of manual input issues.

Full data file: here
Source code of example: here
Run the example: here

Come and learn about it on this course which runs here