revUp - Updates and news for the LiveCode community
Issue 129 | February 23rd 2012 Contact the Editor | How to Contribute

Future Field Features
LiveCode 5.5 has gone into testing, and we have a preview of the new Field Features for you

By Mark Waddingham

If you have a LiveCode Developer Program subscription, you will already know that LiveCode 5.5 dp-1 is available, and hopefully you have downloaded it and started testing. We're waiting for your feedback! We asked Mark Waddingham to tell us exactly what has been done to the LiveCode Field in this important release.

New Field Object

Flagged char ranges
There is a character-level property flagged in fields.

Runs of characters that have this property set to true will be underlined with a dotted red line, flagging them as potentially needing attention. The purpose of this property is to allow easier implementation of on-demand spell-checking in fields.

The flagged property differs from other character properties in that it is considered extra information that is not part of the styling of the field. This means that its value does not appear in, and has no effect on, any of the (persistent) content formats (styledText, rtfText, htmlText) nor will it appear when content is copied or pasted.

To make it easier to handle the flagged property (since it does not appear in any persistent form), there is a field and field chunk level property flaggedRanges. This property returns a return-delimited list of character ranges on which flagged is set to true. The character ranges are relative to the first index of the chunk the range is requested for.

For example, consider the following text (underline indicates the flagged ranges):

Hello LiveCode World
Goodbye LiveCode World

With this we get:

the flaggedRanges of field => 7,14<cr>30,37
the flaggedRanges of line 1 of field => 7,14
the flaggedRanges of line 2 of field => 9,16

The flaggedRanges is both settable and gettable. For example, the following has no resulting effect:

  set the flaggedRanges of line 2 of field 1 to the \
      flaggedRanges of line 2 of field 1

Note: The flagged property is useful as it gives a way of highlighting text which does not in any way effect the content. The flaggedRanges property is useful because it allows you to easily transfer the flagging between fields and portions of fields.

Chunk index conversion
There are two field chunk properties charIndex and lineIndex.

These properties allow easy (and efficient) conversion between the character index at which the beginning of a line sits, and the line index in which a character sits.

The charIndex of a field chunk will return the character offset in the field of the start of the chunk.
The lineIndex of a field chunk will return the line offset in the field of the start of the chunk.

For example, consider the following text:

Hello World
Goodbye World

With this we get:

the charIndex of line 2 of field => 13
the lineIndex of char 7 of field => 1
the charIndex of word 2 of field => 7

Note that the charIndex property takes into account the Unicode nature of the chunk (if applicable) meaning that in some cases the charIndex of char 5 is not necessarily 5.

Note: This is useful because there are many places where the engine gives you a field chunk in char or line form, but you might want the the opposite indices. (e.g. the lineIndex of the selectedChunk gives you the index of the line containing the selection).

Metadata char property
There is a new character-level property metadata in fields.

This property can be set on runs of characters and is intended to allow applications to store data attached to character ranges in fields that has no effect on engine processing.

The value of the metadata property is limited to non-binary strings at present.

Note: This is useful because it allows data to be associated with text in fields that moves automatically as the field is edited. While you could do something like this with the linkText property that already existed, setting the linkText has an effect on both the visual presentation and UI interaction of text in fields.

Individual textStyle manipulation
The textStyle property of fields has been extended with array notation to allow easier manipulation of individual styles.

It is now possible to use expressions of the form:

   the textStyle[<style>] of <chunk> of field

Where <style> is one of bold, condensed, expanded, italic, oblique, box, threedbox, underline, strikeout, link, group.

Such a property can be set to either true or false and will result in either that style being added (true), or removed (false) from the textStyle's of each character in the range.

Getting the property will result in one of true, false, or mixed depending on whether the property is set on all chars in the chunk (true), unset on all chars in the chunk (false), or set on some and unset on others (mixed).

For example, consider the following text where the middle word is in italics:

Hello LiveCode World

Peforming the following command will add bold style to the given range without affecting the setting of italic:

   set the textStyle["bold"] of char 3 to -3 of field to true

Results in:

Hello LiveCode World

Note: This is useful because previously there was no (straightforward) way to set text styles individually. Indeed, the textStyle property is a comma-separated list of styles applied to the character so if you set it to italic, it obliterates the bold setting (if present).

textChanged message
The field will now dispatch a message textChanged whenever a user (or simulated user) action may have caused the content of the field to change.

In particular, the following actions will result in a textChanged message being sent:

  • typing into the field (whether non-Unicode or Unicode text)
  • using the type command to type into a field
  • pasting text into a field (whether via built-in keyboard shortcut, or the paste command)
  • cutting text from fields (whether via built-in keyboard shortcut, or the cut command)
  • drag drop operations on fields

The message is sent immediately after the operation completes, but before a screen update has been requested - the corresponding update will occur at the end of the first command in the textChanged handler. This means that you can lock screen as the first line of the handler to delay the update (allowing you to modify the content of the field without any flicker).

To prevent potential for infinite recurison, calls to textChanged will not nest. That is to say that if a textChanged handler is being executed for a given field, another textChanged message will not be sent to to it should one be triggered.

The textChanged message will be sent after messages such as keyDown and pasteKey but before messages such as keyUp.

Note: This is useful because at the moment there is no simple way to react when something non-script based modifies the field, you have to hook into numerous messages. Indeed, some things (such as Unicode text entry) result in no notification at all.

Unicode-aware field chunks
The field will now (internally) correctly compute character ranges for chunks regardless of whether the text in the chunk is native, Unicode or a mixture.

Essentially this means that you can set and get properties of chunks in fields without having to worry about handling unicode text specially.

For example, consider the following text:

Previously attempts to do things like:

   set the textStyle of word 2 of line 1 of field to "bold"
   set the backColor of line 2 of field to "red"

Would, generally, not work because the field would not take into account the (two-byte) unicode characters correctly.

It should be noted that care still needs to be taken when manipulating the text content of such chunks. The value of a chunk such as:

   line 2 of field 1

Will still be the 'mixed' representation the engine has internally which cannot really be processed in script. If there is need to process the content of chunks then the text and unicodeText properties should be used:

the text of line 2 of field 1 => "Goodbye ????? World" (native text encoding)

Notice that the text property converts the content to the native text encoding (MacRoman, Windows-1252, ISO-8859-1 depending on platform) with '?' used for unconvertable characters; whereas the unicodeText property returns the content of the chunk as (uniform) UTF-16.

Note: This is useful because it enables many operations on fields to be performed regardless of whether the field contains Unicode text.

styledText content format
The field now has a new 'content' format - styledText. This is similar to rtfText and htmlText in that it provides a script-processable representation of the field's content. It differs from these two formats in two ways: it is a fully faithful representation (set the styledText of field to the styledText of field results in no change to the field); it is array-based.

The styledText property returns a numerically-indexed array of paragraphs, each index representing each paragraph in the field in order:

tStyledTextArray[1] = <first paragraph array>

tStyledTextArray[<n>] = <last paragraph array>

Each paragraph array has up to two keys:

tParagraphArray["style"] = <array containing paragraph-level styles>
tParagraphArray["runs"] = <paragraph content array>

The style array contains the values for each of the paragraph styles set on that paragraph. The list of styles that are supported are: textAlign, listStyle, listDepth, listIndent, firstIndent, leftIndent, rightIndent, spaceAbove, spaceBelow, tabStops, backgroundColor, borderWidth, borderColor, hGrid, vGrid, dontWrap, padding.

The paragraph content array is a numerically-indexed array of runs, each index representing each run in the paragraph in order:

tParagraphContentArray[1] = <first paragraph run array>

tParagraphContentArray[<n>] = <last paragraph run array>

Each paragraph run array has up to three keys:

tRunArray["style"] = <array containing character-level styles for the run>
tRunArray["metadata"] = <metadata of the run (if present)>
tRunArray["text" (or "unicodeText")] = <text content of run>

The style array contains the values for each of the characters styles set on that run. The list of styles that are supported are: textFont, textSize, textStyle, textColor, backgroundColor, linkText, imageSource.

If a run has Unicode text in it then the run array will have a "unicodeText" key containing its content encoded as UTF-16. Otherwise, the run array will have a "text" key containing its content encoded in the native text encoding. Note that any ",<tag>" field will be stripped from the textFont when generating a styledText array, and completely ignored when parsing a styledText array.

For example, take the following content consisting of two paragraphs:

Centered Hello World

This would transpire as the following array:

[ For brevity, single element arrays are represented using {...} notation ]

When setting the styledText property, the engine uses a very permissive algorithm to parse the arrays as follows:

   parseStyledTextArray pStyledText
      repeat for each element tEntry of pStyledText
            if tEntry is a sequence then
                  parseStyledTextArray tEntry
            else if tEntry has key "runs" then
               begin paragraph with style tEntry["style"]
               parseStyledTextRunArray tEntry["runs"]
            end paragraph
         else if tEntry is an array then
            append tEntry["text"] with style tEntry["style"]

   parseStyledTextRunArray pRun
      repeat for each element tRun in pRuns
            if tRun is a sequence then
                  parseStyledTextRunArray tRun
                  append tRun["text"] with style tEntry["style"]

Basically, the engine flattens any nested numeric arrays within the tree; then iterates through the result ignoring any empty entries. If an array has a 'runs' key it is treated as an independent paragraph; otherwise it is assumed to be a 'run' and the styled text it contains is appended to the current paragraph. Note that the 'text' field of a run can contain newlines the presence of which will cause a paragraph break at the appropriate points - if such a break is made, the paragraph attributes will be copied across the break.

Note: This property is useful because it provides a easily manipulable representation of the field contents. In particular as it is array-based it allows much easier modification of specific parts of the field than having to manipulate the string-based htmlText and rtfText.

Hierarchical list support
The engine now has support for hierarchical lists in fields. These are controlled with the listStyle, listDepth and listIndent paragraph-level properties.

The listStyle property determines what kind of index is displayed for the paragraph. It can be one of the following: disc, circle, square, decimal, lower latin, upper latin, lower roman, upper roman, skip.

The disc, circle and square settings cause the paragraph to be displayed with a bullet; while the decimal, lower/upper latin and lower/upper roman settings cause the paragraph to be displayed with its index written in the specified way (number, alphabetic or roman numerals). The index of a paragraph is computed as the number of preceeding paragraphs with the same listStyle before a paragraph is encountered with the same or lesser listDepth and different listStyle. In particular, this rule means that indexing continues after nested lists in the way you would expect.

The skip setting causes the paragraph to be displayed with an empty label - such paragraphs are ignored when computing the index of other paragraphs allowing it to be used to display multiple paragraphs in the same list item.

The listDepth property is a number between 1 and 16 which determines the nesting depth of the list. The depth of a list is used to compute the paragraph's list index as well as the indentation (if firstIndent is not set).

The listIndent property determines the amount to indent the content of the paragraph by for each level of depth. If the firstIndent property of the paragraph is set instead, the engine uses firstIndent together with leftIndent to determine the placement of the label and content - in this case, the listDepth is ignored.

For example:

Note: This is useful as it makes it easy to display (and allow the user to edit) nested lists with automatic labelling.

Paragraph level properties
The engine now has support for paragraph-level properties. Some existing field-level properties have been lifted to the paragraph-level, alongside several new properties for controlling the display of paragraphs.

The available paragraph-level properties are as follows: textAlign, listStyle, listDepth, listIndent, firstIndent, leftIndent, rightIndent, spaceAbove, spaceBelow, tabStops, backgroundColor, borderWidth, borderColor, hGrid, vGrid, dontWrap.

The textAlign property allows the alignment of a paragraph to be set independently. It can be one of left, center, right. If the property is not set on a paragraph, the setting is inherited from that of the field.

The firstIndent property determines how much to indent the first line of a paragraph. The indent is considered to be relative to the leftIndent setting of the paragraph so positive values shorten the first line whereas negative values lengthen the first line. [ Note that in the case of a paragraph with non-empty listStyle, the firstIndent property (if set) determines the offset of the label relative to the start of the text and (in that case) must be negative. ]

The leftIndent and rightIndent properties determine how much space to leave at either side of the paragraph before any border and content is rendered.

The spaceAbove and spaceBelow properties determine how much space to leave above and below the paragraph before any border and content is rendered.

The tabStops property allows tab-stops to be set on a per-paragraph basis. If the property is not set on a paragraph, the tabStops setting of the field is used.

The backgroundColor property allows the color of the content area (inside any paragraph border) to be filled (note that strictly speaking this property is not inherited, but the effect is the same as if it was as the background of the field is rendered before the paragraphs are so the background color at the field level will 'show through' to the paragraph if the paragraph has no background color).

The borderWidth property determines the width of the border to draw around the paragraph.

The padding property determines the amount of space between the border and the text content. In vGrid mode, each cell is reduced in width to accomodate the padding at either end of the cell (i.e. it provides padding in each cell in addition to around the line as a whole).

The borderColor property determines the color to use when drawing the border and grid (if either of the grid properties are set). The borderColor is inherited from the field if not set on the paragraph.

The hGrid property controls automatic generation of grid lines between paragraphs. If the hGrid is set to true on a paragraph (or inherits the true setting from the field) then an implicit 1 pixel wide border will be placed both above and below the paragraph. Additionally, the engine will elide borders of adjacent paragraphs which have hGrid set to true if their properties are compatible. Specifically, if two adjacent paragraphs have hGrid set, have the same borderWidth and no (or 0) spaceAbove and spaceBelow, a single pixel grid line will be placed between them rather than the normal border.

The vGrid property allows automatic generation of grid lines between tabs and cell clipping to be set on a per-paragraph basis. If the property is not set at the paragraph level it's value is inherited from the field.

The dontWrap property allows line-wrapping to be set on a per-paragraph basis. If the property is not set at the paragraph level it's value is inherited from the field.

Note: These features are useful for obvious reasons. In particular, the paragraph-level indents, borders and back-colors make it much easier to develop well styled text content; and the paragraph-level hGrid, vGrid and tabStops enable simple tables to be embedded inbetween normal paragraphs.

vGrid column visibility and tabWidths
It is now possible to hide individual columns when vGrid is in effect. The engine will skip any columns which have an effective tabStop width of 0. For example, suppose you want a table-like layout with columns of 50 wide but with the first and third columns hidden. You can use the following tabStops:


This means the first column is of zero width (hidden), the second is 50 wide, the third is of zero width (hidden) then all subsequent columns are 50 wide.

To help with control column visibility a new (synthetic) property tabWidths has been added. This property is similar to the tabStops property except that it returns the tab stops as a sequence of widths rather than as absolute positions. For example, the following are equivalent:

tabStops = 50,100,150,200,250,250
tabWidths = 50,50,50,50,50,0

Note: This feature is useful as it allows 'hidden' information to be stored in simple tables in fields on a per-row basis. The tabWidths property makes controlling visibility of individual columns much easier, as the relevant item can just be replaced with 0.

vGrid fixed width table mode
When vGrid is in effect, if the final tabStop has 0 width the engine treats the paragraph as a fixed width table, the width being the position of the final tabstop.

In this mode, the width of the paragraph is considered to be the (fixed) width and this width is used to do the following:

  • compute the placement of the table in the line taking into account the textAlign property
  • compute the background rectangle to fill with the backColor property
  • compute the placement of the border (if any)

In this mode, rather than borders and background fill going from left margin to right margin, they will instead go from left of the table to right of the table.
For example, suppose you want a table-like layout with only 5 columns of 50 wide. You can use the following tabStops:


This will result in a table of width 250 pixels.

Note: This feature is useful as it allows the number of columns to be explicitly specified (usually tabs continue for the entire width of the field) and then use the explicit width to place the table in the line allowing easy centering of simple tables within fields.

Unicode handling changes
The field object (internally) stores text as a mixture of sequences of bytes representing native encoded text and bytes representing UTF-16 encoded text. As it stands it determines what encoding a particular byte (or pair of bytes) is using by looking at item 2 of the effective textFont property of the run; and the 'text' property (and by extension the 'value' of a field object) is the concatenation of all these bytes. This will be changing in 5.5.

From a conceptual point of view, the encoding of a byte (or pair of bytes) in a field will become an intrinsic property - something not settable by script. Instead, the encoding will be determined at the point the text enters the field - for example setting the unicodeText of a range will enter UTF-16 encoded characters; while setting the text of a range will enter native encoded characters. This is a much more 'correct' way to handle the issue, and will mean scripts updated to work in the new way will work even when the engine is transparently unicode (at that point the 'text' property will happily understand Unicode).

The changes being made are as follows:

  • The textFont property will no longer have (and will ignore when set) any ",<lang>" tag.
  • A read-only 'encoding' property will be added that will return either 'unicode', 'native' or 'mixed' depending on the encoding of the individual chars that make up the chunk.
  • The 'text' property of the field (and by extension the value of a 'field ...' chunk) will return the content of the field in native encoding.
  • The 'styledText' content format will change slightly. Runs will have a 'text' key if the content is encoded natively, or a 'unicodeText' key if the content is encoded in UTF-16.

These changes do mean that scripts that attempt to manipulate unicode text by setting the text and then directly setting the textFont property will need to be updated - they must change to setting the unicodeText in those cases. Note that this should now be much easier as the field will correctly allow things like:

   set the unicodeText of line 4 to 5 of field 4

(See above section on Unicode-aware field chunks).

In dp-1, the 'encoding' property has been added and the 'styledText' has been updated as described above.

The other changes described will appear in dp-2 assuming no problems arise.

If you want to get your hands on these features without waiting for the final release of LiveCode 5.5, you can join the LiveCode Developer Program here and start testing today. LiveCode 5.5 will be shipping shortly, and this upgrade will be included free of charge with any 5.0 purchase in the store from this date.

About the Author

Mark Waddingham is CTO for Runtime Revolution Ltd.


Main Menu

What's New

Buy Tickets to RunRevLive.12 NOW!