Gone Structured!

I have used FM (FrameMaker) since early 90s on Mac OS, Unix and Windows, but did not seriously start going structured until December 2005. I did not have any need for XML as such, but I value a "close relationship" to XML for future purposes. My original reasons for going structured were:

• being able to use certain plug-ins that only work with Structured FrameMaker, most specifically a plug-in to be able to manage conditional chapters in FM books;
• higher level content management/reuse within FM;
• an extremely high degree of consistency and robustness;
• strong means to provide online help documents, via RoboHelp or similar;
• some formatting features can only be had in Structured FM, such as the ability to inherit formatting, to nest formatting, and to do context-dependent formatting, all contributing to a superior look of the documents;
• the future of FM seems to lie in Structured FM and XML;
• if documents were in Structured FM, one would have more choices for alternatives in the future, and it may make interworking easier.

All in all, I am very satisfied with the result, and along the road, I got several more bullets than those listed above. But the process of getting there involved MUCH more work than anticipated!

The beginning

When reading about Structured FrameMaker in the User Guide, the most daunting and intimidating experience was the constantly recurring statement "ask your Application Developer"! It turned out that this precious Application Developer was of course going to be: me!

So, to learn about EDD (Element Definition document), you need to spend many hours of reading bits and pieces in the monster 619 pages "Structure Application Developer's Guide". Some relief, however, is provided by the much newer document "Migrating from Unstructured to Structured FrameMaker". The developer's guide is based on the assumption that anyone doing anything EDD will also do full XML round tripping with read/write rules and what not, also assuming it targets a large corporation with thousands of employees! I think that is a very unnecessary assumption.

It is often claimed that the structure is just like a layer above the existing things. So, my early naïve thinking was of course simply to add that layer. I simply demand that my current formatting shall remain more or less untouched.

However, I have come to the conclusion that the "old" formats in an unstructured template are actually going to be a pretty poor basis for a structured template! If you were to "simply" auto-convert a document to a structured document that maintained formatting through a more or less automatically generated EDD (requires at least FM 7.2), you would not exploit much of the features of a structured environment! And you could not easily at a later stage add features that only structured can offer, since you would have to change the structure, which would invalidate documents made according to the old structure. So: bad idea! I'll describe more on that under "EDD vs format catalogs".

If you want to start experimenting yourself, I recommend that you go to http://www.weststreetconsulting.com and download a little learning package "FrameSLTDocSample_FM7" that includes examples of an unstructured document, a commented EDD, and most importantly: a working conversion table! A big thank you goes to Russ Ward for this! I learned a lot through that procedure. Although, I did not really use anything from that EDD in my final EDD, it was instructional.

I started about November/December 2005, had a pretty good EDD for my manuals about March 2006. This was of course not full time work, since I had to do all the normal work also. It was all "extra" — many late evenings! It was tried and used for "real" work, and improved over the next few months. As always, I discovered a few improvements I wanted to include, so by September one can say it was really mature and well-tried. It has been used for several manuals, and they have also been translated into more than a dozen other languages. The EDD has 7 language-dependent variables that set things like labels for chapter, appendix, figure, table, etc, so I had language versions of the EDD made simply by importing those variables.

Roughly speaking, one could say that it takes a few months of part-time job to make a really good EDD if you start from scratch, half a year or more to make it fully mature and proven for your kind of documents. (But if you work full time with it, or you are several people, it might go much faster.) All this is assuming that the EDD is a pretty general and flexible one, intended for more than a single narrowly defined document type.

Adobe canned templates

I am sorry to say that the templates Adobe provide are mostly crap! They are mostly not illustrative of good techniques! And there are lots of inconsistencies. So beware! I think it is bad teaching.

The only Adobe EDD in the FM directory that had anything at all to teach me concerning EDD design was the "SGMLDevGuide.edd" in 'FrameMaker7.2\samples\fmsgml\'! For instance, it was that EDD that gave me the good hint about having an XRefLabel attribute. There are also other smart things in that EDD. But the really bad thing with it is that it is totally and only targeted for producing that specific "SGML Developers Guide", so you can't really use it for other purposes.

"Application" development?

It is a bit confusing that, while no application is needed to just work with EDDs in Structured FrameMaker, one needs to work carefully with the manual "Structure Application Developer's Guide", in order to write and modify EDDs! Thus, you are regarded as an "Application Developer" just to make good templates, even if those are intended only for one person!? Yet, you do not need to apply an application! That's a bit confusing, but might be acceptable ones you know about it.

EDD vs format catalogs

Then we have the anxiety-ridden decision about what to place in the EDD and what to place in the paragraph/character catalogs:

Historically, while working in unstructured FM, I have numerous times had to change both fonts and language in far too many formats! I deeply hate it! For each of these changes, I have been forced to painstakingly go through every single paragraph format, because there is no inheritance at all! It is a nightmare to go through each time if you have many formats! And there is always something I miss in the first round! Avoid at all costs!

The language issue is especially important if you work with manuals that are to be translated into many languages! I personally only author in three different languages: US English, British English, Swedish. But when documents shall be translated into several languages, there MUST be an easy way to change the language so that hyphenation becomes reasonable. And for certain languages you also have to change fonts. With FM 7, which wasn't even Unicode-aware, any non-western language had to be given a suitable font for the language category! I created "language files" that had these adapted paragraph formats, so I could import them into the translated documents.

So, the implications of the above say that the paragraph catalog should be kept short, but also that both language and font should be set in the paragraph catalog and not the EDD.

I've also had to change the width of side heads several times. But if you change the width of the side head, the indentation and tabs of paragraphs that span the column into the side head will also have to be changed! So, in conclusion: let the EDD do that work; specify the size of the side head, the column width, and any special indentation in variables of the EDD so it is very easy to change and maintain consistency!

Reuse of the EDD: It is desirable to be able to use the same EDD for as large a category of documents as possible, although it is okay to make small variations of the EDD by importing different variables to support completely different layouts, such as different size of side heads. Fonts for title and various headings may be different in different kinds of documents or different organizations, and it may change over time.

The guiding principles for EDD vs format catalogs

The above considerations led me to the following conclusions, or guiding principles:

Use paragraph formatting only where there might be a future need to change fonts, or to change the language. Never specify font family names in the EDD! But let the EDD do the rest, unless it can refer to a named character format, since those can have several properties set to As Is, which means that a character format can act as a "format modifier", only changing a single property.

(I did consider having a language attribute though, so that I could easily change the language for the whole document, but I did not do that since I would have to list every conceivable language, and perhaps worst: since some languages require another font anyway, it would cause a font dependency!)

The result of the guiding principles

The above principles make it possible to use the EDD for a relatively wide range of document types, since it is very flexible.

I have a paragraph catalog of only about 20 items. Of course there is Body, and CellBody since they usually use different fonts. Equation, FigureCaption, TableTitle, Author and Title. A Header, and separate paragraph formats for all the headings, since, in a new type of document, headings may very well need new fonts (and sizes).

The character format catalog, on the other hand, is much longer — most of them have several properties set to As Is, so they are not as cumbersome to maintain. But the formats that are used to set text lines in graphics need to be specified both to their fonts and sizes, so they need to have the language changed if they ever will be spell checked, and for certain languages you may also need to change fonts.

Since the EDD has a variable for the size of the side head, it is relatively easy to change the whole layout by changing the few paragraph formats and import a few variables into the EDD. The size of the side head is used to specify various indents for stuff that span the side head and columns.

The really nice consequence of the guiding principles is that you can make the formatting context sensitive! That makes the whole formatting of the document so much nicer!
Examples:

• A side head usually has some distance to the previous paragraph, but not so much. However, if that side head comes right after a section head, it is very ugly if the side head sits too close to the section head, so it really needs some more space! An EDD can do that automatically for you!
• Lists: It is of absolutely vital importance that all indentation, spacing, font size reduction are controlled by the EDD, since that is what makes it possible to nest lists arbitrarily! Formatting of lists will be fully automatic for any nesting, and attributes can control their appearances!
• Text ranges: Also here it is important to set the point size reduction (for texts that need it) in relative terms, so that the element can be placed in different contexts and still maintain the correct relative size!

Tables

The worst obstacle I stumbled into immediately was the complete lack of control over tables from the EDD. No way to control positioning, set float/not float, orphans, or anything else! The only thing you can control is whether to let the table-format or the EDD control the para formats, and you can actually control the alignment of those paragraphs! Well, that's better than nothing, but…

The issue was complicated by the fact that, up until now, I have mostly used floating tables. I simply demand that I should be able to use both float and non-float tables, but I have come to the conclusion that, one should use separate elements for them! That seems absurd, but it's necessary! Actually, float tables don't fit very well with the current EDD standard.

Examples of table problems:

There are several reasons why you need a special Anchor paragraph (2pt with -2pt below). One reason is that it is the only way to control pagination (e.g top of next column/page) from the EDD via an attribute. Another reason is: if you were to place the table (without anchor para) just after a section heading that spans the column+side head, it would NOT be positioned in the column! So if you move such a table from after a normal para to just after a heading, you would be very surprised (and annoyed)! Even worse: if that table fits on the page, but not the following paragraph, it would move all of the heading, the table and the following para to the next page, since the heading will always have 'Keep with next'! Super annoying! Moreover, if the table is wider than the column, but anchored in a "column para", and you set the table to start Top of Column, then nothing happens! But if you set it to start Top of Page, then it will not only move to the next page, but also the paragraph it was anchored to will move, but to a separate page!! Jeeezes! If you instead set Orphan to 99, the paragraph and the table will both move to the next page!

The above shows that you really must use a special anchor para, and that should be set to 2pt, and have a below of -2pt. All those tables should then have a Space above of -2pt! Effectively, the anchor disappears! Space above the table can be set by an attribute that sets the space above the anchor. The anchor should NOT be a separate element — the container element will set the anchor paragraph, and after that the actual table. I recommend that the line spacing should also be 2pt. If you do this scheme, you can even place a side head to the left of the table if you want! That's neat! It also allows an attribute to control how the table spans the page: for instance, if the anchor is set to go just in the side head, the table will also be confined to the side head only (if it fits)! Also neat! Or you may want it to span all columns and side heads.

But if you use a float table, the above scheme is bad! One example: If the float table does not fit on the page, and you had a special Anchor para, the table moves to the next page, but the anchor will remain as white obnoxious space before that next "normal" paragraph that slips in before the table! Ugly! If not to say completely unacceptable! It could be ever worse: if the following para is another section, it may turn out that the table moves into the next section! That can be confusing! Especially in a structured flow, you would not expect that.

The conclusion is that one must have two different elements: one for float tables, where you need to set the space above in the table, and the other is a container that starts with an anchor and then a table. The latter scheme allows much more control through attributes that sets various things in the anchor.

Overall, it works very well! But it is of course inelegant that there is a dependency between the element and what table format is allowed for that table — something the EDD cannot enforce.

Figures

To my experience in regular FM, using a table has hitherto been the only way to consistently handle figures and their captions to be grouped, placed (e.g by 'float'), referenced, and formatted without problems. In structured FM, it also adds a very important advantage: when cross-referencing the figure, you cross-reference the entire table, and you will automatically get the table title since it is defined as the first child! This means that when you make a PDF, the hypertext link will be perfect, since when you click on the link, it will scroll to the table (i.e the figure) and not to the caption that usually sits below the figure! Great! So: Always use tables for figures that will have captions!

Just as for "normal" tables, I need separate elements for figure tables that include an invisible anchor, and floating figures without such an anchor.

The result!

This EDD can be used for a very wide range of documents, although they all are targeted towards manuals, instructions and reports:

• Single-file documents
• Multifile books
• A single multilingual book
• The same EDD for FrontMatter, preface, chapters, appendix

Numbering of tables and figures automatically changes according to the chosen topmost element, making it extremely flexible: Books number them according to chapter, like Figure 1-2, Figure 1-3, Figure A-1, Figure A-2, …, whereas single-file documents (using Document as the topmost element) will number them sequentially like Figure 1, Figure 2, Figure 3, ... Preface will not number its sections, and it uses simple numbering of any figures/tables. Appendix items will be numbered as Chapter, but with ABC (like Table A-1). Whenever the HLE is changed, the numbering changes accordingly! A Book can actually use the element SingleChapter instead of Chapter, which is very useful for a multilingual book: It may (or may not, depending on an attribute choice) number the two levels of sections under it, but it will not number the SingleChapter itself. Tables and Figures will be simple-numbered according to "Table 1", "Figure 1". Each such "chapter" can then be in a separate language, and each language will thus have the same section/figure/table numbers as other languages! And all these variants are controlled by the EDD — not formats in the template file! So you can copy and paste and move and change elements and get consistent changes! The different files use the same paragraph catalog!! This is really what facilitates reuse!

Lists can be nested arbitrarily! Mixing numbered and un-numbered lists is OK! Attributes control the appearance. E.g: set roman bold numbering for a large long list that has numbered child lists.

Automatic symbols in the side head: Using reference page graphics and text warnings, a note can get the appropriate symbol/text to the left of the paragraph. An attribute controls what kind of note/warning it is. (Rendering in FM, however, is lousy, so I have to hit ctrl-L every now and then while editing such a note.)

Generated files

Since I have seen on forums that some people seem to have had problems getting the book structure valid when there are unstructured generated files, I'll give this tip: Generated files are just text, and they become the generic BOOK-COMPONENT element, so just include <TEXT> in the general rule! An example: The element TableOfContents may be <TEXT>, …, which makes the BOOK-COMPONENT file valid as a child to TableOfContents. My general rule has many other elements after <TEXT>, but that is just because there are some cases (single file documents) where I may want to include the TOC into the flow of the rest of the content, and therefore in that case want it structured. But when the TOC is part of a book (as a self-contained file), I see no reason to structure it; I get the formatting I want, and the title stays in the template the way I want. Likewise, I do not structure Master Pages, since I do not see what good that would do.

Overrides?

Don't ever do any overrides to the formatting done by the EDD! Not ever! You must always be able to re-import the EDD and correct any mis-formatting. Don't ever do overrides to the para and character formats! However, it is pretty ok to override table formats, since you rarely have to re-import table formats. I have attributes that do what otherwise would have required overrides, such as start on next page/column, keep with next/previous, left/center/right alignment. That's a real relief to me! No cumbersome "invisible" overrides!

To do some ad hoc formatting, one can use a table. An attribute can control whether to use paragraph formats defined by the table or formats defined by the EDD, and in this case one can choose formats defined by the table. To really do some ad hoc formatting: set up a new text frame (perhaps even inside an anchored frame) and do anything you like in that frame!

Satisfaction guaranteed?

It is a relief working in this structured environment as compared to the regular unstructured FM! What a delight! Just think about being able to copy and paste with absolutely no problems whatsoever concerning unwanted junk formatting! Doesn't matter if it comes from Word or another FM document! With a free plug-in, I just hit 'Esc Q W' to fix the formatting! After I get documents from the translator, I wipe any overrides, strange fonts, etc, by importing files that have formats for the particular language and an EDD for that language. (The EDDs are language dependent in that they set some labels, but those labels are variables that I can import.)

Just think about the enormous flexibility with lists — it is just vastly superior over what anyone can achieve with regular unstructured FM! And being able to change some top level element and automatically get ALL numbering changed! As an example with a multilingual document I was working on, I changed my mind, so I changed all Chapter elements to SingleChapter (in one search&replace), and every numbering everywhere changed consistently! Try that in InDesign or Word or ...!

Just think about having the same template file for Preface, Chapter and Appendix! Therefore much less to maintain!

Cross-reference a table (not the table title) and you get the title/caption automatically! Move a section to a subsection, and it changes automatically to remain consistent!

For those of you who have been using Mac OS X a lot, you know the feeling of going back to Mac OS 9 for a certain job! Same feeling when going back to unstructured FM for a certain job! Perhaps a strange metaphor, since I work in XP while working with FM, but it nevertheless indicates the feeling... the feeling of visiting something old and obsolete...