Main Content

Searching a Json or XML structure for a specific key / value pair in Python

Archive - Originally posted on "The Horse's Mouth" - 2016-10-30 09:27:58 - Graham Ellis

With data loaded from JSon or XML structures, you'll often want to search for attributes by element name - without having to code the structure, and with the code adapting itself to mean subtle structure changes in the suppied data.

Here's an example written during lat week's tailored Intermediate Python course ... taking a Json object (loaded from our web resource) and displaying a list of all the elements of a specific name.

As good python code encapsulate all the hard work into methods, our main progam is very short.

If I want to find the value of all the fields keyed "where" ... Loading the data

  fh = open("jflow.json")
  igot = json.load(fh)
  found = parseDict(igot,"where")

and the hard work is done in parseDict as json.load returns a dict. parseDict has been written to return a list.

Sorting the results:

  if sys.version_info < (3,0):
    found.sort(lambda x,y:cmp(len(x),len(y)))
  else:
    found.sort(key = lambda x:len(x))

and displaying the results, 4 per line.

  print("Here are the remote IP visitors to your images")
  for k in range(len(found)):
    sep = ((k+1)%4 and k+1 != len(found)) and " " or "\n"
    sys.stdout.write("%16s%s"%(found[k],sep))

Full source code is [here] . The data ('pretty printed' as Json compressed is hard to read!) is [here]

parseDict is a piece of recursive code - two re-entrant pieces of code which call each other as they traverse your structure and find lists and dicts within other lists and dicts. I am not normally a fan of recursive / re-entrant code on courses, as they're typically over-clever solutions looking for problems, but this particular use is sensible.

I'll cover this for you on Python Intermediate courses or on Python Programming courses (in that case after class hours if you make a specific request, as not everyone wants to do this).