HTML parser

[Bob Nemec]

Is there a simple way to parse HTML in VA?

I can find HTML parsers for VW and Squeak, and SgmlParser in VA looks interesting... but I figure I'd ask before diving in head first.

Any suggestions?

[solveig]

Bob,

The SGML parser in the product might suit your needs, but it might be too strict.

What do you want to parse?

Solveig

[Bob Nemec]

I'm trying to parse saved web pages. I'm using an iMacro script (http://www.iopus.com/) to download pages from a financial site. I then want to pull data out of the page.

iMacro can do most of the parsing, but I'd like to position my app to eventually be 100% Smalltalk. The web browser scripting will probably be tricky, but being able to deal with the raw page data is a first step.

I'm doing this as an add-on exercise to Rob Vens' MijnGeld. To get started, my code is VA domain classes only (no UI). My plan is to automatically download a person's daily bank transactions and email them to the user, along with current budget status info.

I've got the download and email working (with iMacro and VA to do the emailing). Next is the budget and transaction hooks.

[tc]

Hi Bob,

The xml parser in smalltalk uses an sgml parser underneath to parse out xml. HTML is similar but allows many things that are not allowed in XML. A few examples are:

1. HTML start tags may not have end tags.
2. Start and end tags might overlap other start and end tags rather then nest.
3. Attributes may not have quotes.
4. The HTML page might not have a single root element.
5. Things like 'A&P' in HTML would have to be 'A&P' in XML.

. . . I am sure the st xml parser would complain about some of these things. If you know the data you are after in the HTML, maybe using the st string functions would work better?

--tc

[quixotik]

Or one could try to run that html, assuming it's not well-formed to begin with, through HTML tidy (http://tidy.sourceforge.net/) and then use the sgml parser? Just a thought.

Robin

[tc]

Hi Robin,

Good suggestion. Thanks.

--tc