Clean up your Web pageswith HTML TIDY
Clean up your Web pagesWhen editing HTML it's easy to make mistakes. Wouldn't it be nice if there was a simple way to fix these mistakes automatically and tidy up sloppy editing into nicely layed out markup? Well now there is! Dave Raggett's HTML TIDY is a free utility for doing just that. It also works great on the atrociously hard to read markup generated by specialized HTML editors and conversion tools, and can help you identify where you need to pay further attention on making your pages more accessible to people with disabilities.
Tidy is able to fix up a wide range of problems and to bring to your attention things that you need to work on yourself. Each item found is listed with the line number and column so that you can see where the problem lies in your markup. Tidy won't generate a cleaned up version when there are problems that it can't be sure of how to handle. These are logged as "errors" rather than "warnings".
Dave Raggett has now passed the baton for maintaining Tidy to a group of volunteers working together as part of the open source community at Source Forge. The source code continues to be available under an open source license, and you are encouraged to pass on bug reports and enhancement requests at http://tidy.sourceforge.net.
More recently, Tidy has been extended to support HTML5 and to clean up HTML exported from Google Docs. The source code is available on github, see tidy-html5.
If you find HTML Tidy useful and you would like to say thanks, then please send me a (paper) postcard or other souvenir from the area in which you live along with a few words on what you are using Tidy for. It will be fun to map out where Tidy users are to be found! My postal address is given at the end of this file.
The W3C public email list devoted to HTML Tidy is: <[email protected]>. To subscribe send an email to [email protected] with the word subscribe in the subject line (include the word unsubscribe if you want to unsubscribe). The archive for this list is accessible online. If you would like to contact the developers, or you just want to submit an enhancement request or a bug report, please visit http://tidy.sourceforge.net.
Tidy can now perform wonders on HTML saved from Microsoft Word 2000! Word bulks out HTML files with stuff for round-tripping presentation between HTML and Word. If you are more concerned about using HTML on the Web, check out Tidy's "Word-2000" config option! Of course Tidy does a good job on Word'97 files as well!
Tidy features in an article by Scott Nesbitt on webreview.com, and more recently on Dave Central's Best of Linux, and as tool of the month on Unix Review by Joe Brockmeier, who writes:
"One thing I love about the UNIX philosophy is the idea that each program should do one job and do it really well. There are zillions of small tools for UNIX-type OSes that make life much easier and are hugely useful, but they don't necessarily get written about. They certainly don't receive the same kind of coverage that Apache and Sendmail receive. One of my favorites, HTML Tidy, is a tool for HTML/Web development that I think will interest a lot of folks. HTML Tidy cleans up HTML produced by WYSIWYG editors and such."
Tidy is available as a downloadable binary, as source code (ANSI C), or as an online service at W3C, Info Network, HTML Help's site Valet and other sites.
If you are just starting off and would like to know more about how to author Web pages, you may find my guide to HTML and CSS helpful. Please send me feedback on this, and I will do my best to further improve it.
A new relative to Tidy, HTML Slidy is my open source presentation tool for slides written in XHTML. It provides an accessible alternative to traditional presentation tools like Microsoft PowerPoint. Best of all, there is no software to install, it just works from the Web!
Tidy corrects the markup in a way that matches where possible the observed rendering in popular browsers from Netscape and Microsoft. Here are just a few examples of how TIDY perfects your HTML for you:
<h1>heading <h2>subheading</h3>
is mapped to
<h1>heading</h1> <h2>subheading</h2>
<p>here is a para <b>bold <i>bold italic</b> bold?</i> normal?
is mapped to
<p>here is a para <b>bold <i>bold italic</i> bold?</b> normal?
<h1><i>italic heading</h1> <p>new paragraph
In Netscape and Internet Explorer this causes everything following the heading to be in the heading font size, not the desired effect at all!
Tidy maps the example to
<h1><i>italic heading</i></h1> <p>new paragraph
<i><h1>heading</h1></i> <p>new paragraph <b>bold text <p>some more bold text
Tidy maps this to
<h1><i>heading</i></h1> <p>new paragraph <b>bold text</b> <p><b>some more bold text</b>
<h1><hr>heading</h1> <h2>sub<hr>heading</h2>
Tidy maps this to
<hr> <h1>heading</h1> <h2>sub</h2> <hr> <h2>heading</h2>
<a href="#refs">References<a>
Tidy maps this to
<a href="#refs">References</a>
<body> <li>1st list item <li>2nd list item
is mapped to
<body> <ul> <li>1st list item</li> <li>2nd list item</li> </ul>
Tidy inserts quote marks around all attribute values for you. It can also detect when you have forgotten the closing quote mark, although this is something you will have to fix yourself.
Tidy has a comprehensive knowledge of the attributes defined in the HTML 4.0 recommendation from W3C. This often allows you to spot where you have mistyped an attribute or value.
Tidy will even work out which version of HTML you are using and insert the appropriate DOCTYPE element, as per the W3C recommendations.
This is something you then have to fix yourself as Tidy is unsure of where the > should be inserted.
You can choose which style you want Tidy to use when it generates the cleaned up markup: for instance whether you like elements to indent their contents or not. Several people have asked if Tidy could preserve the original layout. I am sorry to say that this would be very hard to support due to the way Tidy is implemented. Tidy starts by building a clean parse tree from the source file. The parse tree doesn't contain any information about the original layout. Tidy then pretty prints the parse tree using the current layout options. Trying to preserve the original layout would interact badly with the repair operations needed to build a clean parse tree and considerably complicate the code.
Some browsers can screw up the right alignment of text depending on how you layout headings. As an example, consider:
<h1 align="right"> Heading </h1> <h1 align="right">Heading</h1>
Both of these should be rendered the same. Sadly a common browser bug fails to trim trailing whitespace and misaligns the first heading. HTML Tidy will protect you from this bug, except when you set the indent option to "yes".
Setting the indent option to yes can also cause problems with table layout for some browsers:
<td><img src="foo.gif"></td> <td><img src="foo.gif"></td>
will look slightly different from:
<td> <img src="foo.gif"> </td> <td> <img src="foo.gif"> </td>
You can avoid such quirks by using indent: no or indent: auto in the config file.
Tidy offers you a choice of character encodings: US ASCII, ISO Latin-1, UTF-8 and the ISO 2022 family of 7 bit encodings. The full set of HTML 4.0 entities are defined. Cleaned up output uses HTML entity names for characters when appropriate. Otherwise characters outside the normal range are output as numeric character entities. Tidy defaults to assuming you want the output to be in US ASCII. Tidy doesn't yet recognize the use of the HTML meta element for specifying the character encoding.
Tidy offers advice on accessibility problems for people using non-graphical browsers. The most common thing you will see is the suggestion you add a summary attribute to table elements. The idea is to provide a summary of the table's role and structure suitable for use with aural browsers.
Many tools generate HTML with an excess of FONT, NOBR and CENTER tags. Tidy's -clean option will replace them by style properties and rules using CSS. This makes the markup easier to read and maintain as well as reducing the file size! Tidy is expected to get smarter at this in the future.
Some pages rely on the presentation effects of isolated <p> or </p> tags.Tidy deletes empty paragraph and heading elements etc. The use of empty paragraph elements is not recommended for adding vertical whitespace. Instead use style sheets, or the <br> element. Tidy won't discard paragraphs only containing a nonbreaking space
You can teach Tidy about new tags by declaring them in the configuration file, the syntax is:
new-inline-tags: tag1, tag2, tag3 new-empty-tags: tag1, tag2, tag3 new-blocklevel-tags: tag1, tag2, tag3 new-pre-tags: tag1, tag2, tag3
The same tag can be defined as empty and as inline or as empty and as block.
These declarations can be combined to define an a new empty inline or empty block element, but you are not advised to declare tags as being both inline and block!
Note that the new tags can only appear where Tidy expects inline or block-level tags respectively. This means you can't (yet) place new tags within the document head or other contexts with restricted content models. So far the most popular use of this feature is to allow Tidy to be applied to Cold Fusion files.
Tidy is somewhat aware of the preprocessing language called ASP which uses a pseudo element syntax <% ... %> to include preprocessor directives. ASP is normally interpreted by the web server before delivery to the browser. JSTE shares the same syntax, but sometimes also uses <# ... #>. Tidy can also cope with another such language called PHP, which uses the syntax <?php ... ?>
Tidy will cope with ASP, JSTE and PHP pseudo elements within element content and as replacements for attributes, for example:
<option <% if rsSchool.Fields("ID").Value
= session("sessSchoolID")
then Response.Write("selected") %>
value='<%=rsSchool.Fields("ID").Value%>'>
<%=rsSchool.Fields("Name").Value%>
(<%=rsSchool.Fields("ID").Value%>)
</option>
Note that Tidy doesn't understand the scripting language used within pseudo elements and attributes, and can easily get confused. Tidy may report missing attributes when these are hidden within preprocessor code. Tidy can also get things wrong if the code includes quote marks, e.g. if the example above is changed to:
value="<%=rsSchool.Fields("ID").Value%>"
Tidy will now see the quote mark preceding ID as ending the attribute value, and proceed to complain about what follows. Note you can choose whether to allow line wrapping on spaces within pseudo elements or not using the wrap-asp option. If you used ASP, JSTE or PHP to create a start tag, but placed the end tag explicitly in the markup, Tidy won't be able to match them up, and will delete the end tag for you. So in this case you are advise to make the start tag explicit and to use ASP, JSTE or PHP for just the attributes, e.g.
<a href="<%=random.site()%>">do you feel lucky?</a>
Tidy allows you to control whether line wrapping is enabled for ASP, JSTE and PHP instructions, see the wrap-asp, wrap-jste and wrap-php config options, respectively.
I regret that Tidy does not support Tango preprocessing instructions which look like:
<@if variable_1='a'>
do something
<@else>
do nothing
</@if>
<@include <@cgi><@appfilepath>includes/message.html>
Tidy supports another preprocessing syntax called "Tango", but only for attribute values. Adding support for pseudo elements written in Tango looks as if it would be quite tough, so I would like to gauge the level of interest before committing to this work.
XML processors compliant with W3C's XML 1.0 recommendation are very picky about which files they will accept. Tidy can help you to fix errors that cause your XML files to be rejected. Tidy doesn't yet recognize all XML features though, e.g. it doesn't understand CDATA sections or DTD subsets.
Indenting the content of elements makes the markup easier to read. Tidy can do this for all elements or just for those where it's needed. The auto-indent mode has been used below to avoid indenting the content of title, p and li elements:
<html>
<head>
<title>Test document</title>
</head>
<body>
<p>para which has enough text to cause a line break,
and so test the wrapping mechanism for long lines.</p>
<pre>
This is
<em>genuine
preformatted</em>
text
</pre>
<ul>
<li>1st list item</li>
<li>2nd list item</li>
</ul>
<!-- end comment -->
</body>
</html>
Indenting the content does increase the size of the file, so you may prefer Tidy's default style:
<html>
<head>
<title>Test document</title>
</head>
<body>
<p>para which has enough text to cause a line break,
and so test the wrapping mechanism for long lines.</p>
<pre>This is
<em>genuine
preformatted</em>
text
</pre>
<ul>
<li>1st list item </li>
<li>2nd list item</li>
</ul>
<!-- end comment -->
</body>
</html>
tidy [[options] filename]*
HTML tidy is not (yet) a Windows program. If you run tidy without any arguments, it will just sit there waiting to read markup on the stdin stream. Tidy's input and output default to stdin and stdout respectively. Errors are written to stderr but can be redirected to a file with the -f filename option.
I generally use the -m option to get tidy to update the original file, and if the file is particularly bad I also use the -f option to write the errors to a file to make it easier to review them. Tidy supports a small set of character encoding options. The default is ASCII, which makes it easy to edit markup in regular text editors.
For instance:
tidy -f errs.txt -m index.html
which runs tidy on the file "index.html" updating it in place and writing the error messages to the file "errs.txt". Its a good idea to save your work before tidying it, as with all complex software, tidy may have bugs. If you find any please let me know!
Thanks to Jacek Niedziela, The Win32 executable for tidy is now able to use wild cards in filenames. This utilizes the setargv library supplied with VC++.
Tidy writes errors to stderr, and won't be paused by the more command. A work around is to redirect stderr to stdout as follows. This works on Unix and Windows NT, but not on other platforms. My thanks to Markus Wolf for this tip!
tidy file.html 2>&1 | more
To get a list of available options use:
tidy -help
You may want to run it through more to view the help a page at a time.
tidy -help | more
Input and Output default to stdin/stdout respectively. Single letter options apart from -f may be combined as in: tidy -f errs.txt -imu foo.html
Tidy now supports a configuration file, and this is now much the most convenient way to configure Tidy. Assuming you have created a config file named "config.txt" (the name doesn't matter), you can instruct Tidy to use it via the command line option -config config.txt, e.g.
tidy -config config.txt file1.html file2.html
Alternatively, you can name the default config file via the environment variable named "HTML_TIDY". Note this should be the absolute path since you are likely to want to run Tidy in different directories. You can also set a config file at compile time by defining CONFIG_FILE as the path string, see platform.h.
You can now set config options on the command line by preceding the name of the option immediately (no intervening space) by "--", for example:
tidy --break-before-br true --show-warnings false
The full set of options available on recent versions of HTML Tidy can be found in this Quick Reference maintained as part of the Source Forge HTML Tidy project.
This is just an example to get you started.
// sample config file for HTML tidy indent: auto indent-spaces: 2 wrap: 72 markup: yes output-xml: no input-xml: no show-warnings: yes numeric-entities: yes quote-marks: yes quote-nbsp: yes quote-ampersand: no break-before-br: no uppercase-tags: no uppercase-attributes: no char-encoding: latin1 new-inline-tags: cfif, cfelse, math, mroot, mrow, mi, mn, mo, msqrt, mfrac, msubsup, munderover, munder, mover, mmultiscripts, msup, msub, mtext, mprescripts, mtable, mtr, mtd, mth new-blocklevel-tags: cfoutput, cfquery new-empty-tags: cfelse
If you want to run Tidy from a Perl or other scripting language you may find it of value to inspect the result returned by Tidy when it exits: 0 if everything is fine, 1 if there were warnings and 2 if there were errors. This is an example using Perl:
if (close(TIDY) == 0) {
my $exitcode = $? >> 8;
if ($exitcode == 1) {
printf STDERR "tidy issued warning messages\n";
} elsif ($exitcode == 2) {
printf STDERR "tidy issued error messages\n";
} else {
die "tidy exited with code: $exitcode\n";
}
} else {
printf STDERR "tidy detected no errors\n";
}
The latest versions of the source code can be found at the Source Forge developer's site for Tidy, see http://tidy.sourceforge.net.
I would like to thank the many people who have written to me with suggestions for improvements or reporting bugs. Your help has been invaluable.
Jonathan Adair, Drew Adams, Osma Ahvenlampi, Carsten Allefeld, Richard Allsebrook, Jacob Sparre Andersen, Joe D'Andrea, Jerry Andrews, Bruce Aron, Takuya Asada, Edward Avis, Carlos Piqueres Ayela, Nick B, Chang Hyun Baek, Nick B, Denis Barbier, Chuck Baslock, Christer Bernerus, David J. Biesack, John Bigby, Yu Jian Bin, Alexander Biron, Keith Blakemore-Noble, Eric Blossom, Berend de Boer, Ochen M. Braun, Dave Bryan, David Brooke, Andy Brown, Keith B. Brown, Andreas Buchholz, Maurice Buxton, Jelks Cabaniss, John Cappelletti, Trevor Carden, Terry Cassidy, Mathew Cepl, Kendall Clark, Rob Clark, Jeremy Clulow, Dan Connolly, Larry Cousin, Ken Cox, Luis M. Cruz, John Cumming, Ian Davey, Keith Davies, Ciaran Deignan, David Duffy, Emma Duke-Williams, Tamminen Eero, Bodo Eing, Peter Enzerink, Baruch Even, David Fallon, Claus André Färber, Stephanie Foott, Darren Forcier, Martin Fouts, Frederik Fouvry, Rene Fritz, Stephen Fuqua, Martin Gallwey, Pete Gelbman, Francisco Guardiola, David Getchell, Michael Giroux, Davor Golek, Guus Goos, Léa Gris, Rainer Gutsche, Kai Hackemesser, Juha Häikiö, David Halliday, Johann-Christian Hanke, Vlad Harchev, Shane Harrelson, Andre Hinrichs, Bjoern Hoehrmann, G. Ken Holman, Bill Homer, Olaf Hopp, Craig Horman, Jack Horsfield, Nigel Horspool, Pao-Hsi Huang, Stuart Hungerford, Marc Jauvin, Rick Jelliffe, Peter Jeremy, Craig Johnson, Charles LaFountain, Steven Lobo, Zdenek Kabelac, Michael Kay, Jeffery Kendall, Axel Kielhorn, Konstantinos Kleisouris, Johannes Koch, Daniel Kohn, Rudy Kohut, Allan Kuchinsky, Volker Kuhlmann, Michael LaStella, Johnny Lee, Steve Lee, Tony Leneis, Nick Leverton, Todd Lewis, Dietmar Lippold, Gert-Jan C. Lokhorst, Murray Longmore, John Love-Jensen, Satwinder Mangat, Carole Mah, Anton Marsden, Bede McCall, Shane McCarron, Thomas McGuigan, Ian McKellar, Al Medeiros, Chris Nappin, Ann Navarro, Jacek Niedziela, Morten Blinksbjerg Nielsen, Kenichi Numata, Allan Odgaard, Matt Oshry, Gerald Oskoboiny, Paul Ossenbruggen, Ernst Paalvast, Christian Pantel, Dimitri Papadopoulos, Rick Parsons, Steven Pemberton, Daniel Persson, Lee Anne Phillips, Xavier Plantefeve, Karl Prinz, Andy Quick, Jany Quintard, Julian Reschke, Stephen Reynolds, Thomas Ribbrock, Ross L. Richardson, Philip Riebold, Erik Rossen, Dan Rudman, Peter Ruevski, Christian Ruetgers, Klaus Johannes Rusch, John Russell, Eric Schindler, J. Schlauch, Christian Schüler, Klaus Alexander Seistrup, Jim Seymour, Kazuyoshi Shimizu, Geoff Sinclair, Jo Smith, Mike Smith, Paul Smith, Steve Spilker, Rafi Stern, Jacques Steyn, Michael J. Suzio, Zac Thompson, Eric Thorbjornsen, Oren Tirosh, John Tobler, Omri Traub, Loïc Trégan, Jason Tribbeck, Simon Trimmer, Steffen Ullrich, Stuart Updegrave, Charles A. Upsdell, Jussi Vestman, Larry W. Virden, Daniel Vogelheim, Nigel Wadsworth, Jez Wain, Randy Waki, Paul Ward, Neil Weber, Bertilo Wennergren, Yudong Yang, Jeff Young, Edward Zalta, Johannes Zellner, Christian Zuckschwerdt
35 Frome Road
Bradford on Avon
Wiltshire
BA15 2EA
United Kingdom
Dave Raggett <[email protected]> is a consultant with Canon, and previously worked with Openwave, and before that with Hewlett Packard's UK Laboratories. Dave works on assignment to the World Wide Web Consortium, where he is the W3C lead for Voice and Multimodal.