PyKHTML requires PyKDE 3 (and hence in turn PyQt 3 + KDE libs). If you would like to run PyKHTML on servers without an X display then Xvfb is required. Fortunately these requirements should come bundled with most modern Linux distributions, and support for Windows/Mac should appear in the next few months.
Show me some code
Okay. Here is an example (one of many examples included in the bundle) that scrapes the title and navigation from this page, with excessive commenting to give you a feel of what programming with PyKHTML is like:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
import pykhtml PyKHTMLUrl = "http://paul.giannaros.org/pykhtml" def extractBitsFromPage(browser): # getElementsByTagName returns a generator, so we convert # to a list and access the first element title = list(browser.document.getElementsByTagName("title")) print "Title:", title.text # Get the text of the navigation items navigation =  # First get the container of the list items... navigationElement = browser.document.getElementById("navigation") # ... and then loop over the li elements we find for listItem in navigationElement.getElementsByTagName("li"): # Inside the list item is an anchor anchor = listItem.children # And the text inside the anchor is what we want navigation.append(anchor.text) print "Navigation:", " | ".join(navigation) # Stop here, we're done pykhtml.stopEventLoop() def main(): browser = pykhtml.Browser() # the browser is passed as a parameter to extractBitsFromPage # when it is called (when the page has loaded) browser.load(PyKHTMLUrl, extractBitsFromPage) # kick things off pykhtml.startEventLoop() main()
Note of Thanks
Gambit Research, a software company in West London, sponsor PyKHTML development.