USING FILTERS
Multi-threaded NewsWatcher

Contents

 See the page on creating and editing filters.
See the page on regular expressions.

FILTER BASICS

What are filters?

 Filters enable you to hilite, or kill certain articles based on a set of criteria that you specify, and provide a very useful way of drawing your attention to articles that might be of interest, and in filtering out those you certainly don't want to see ("$$$ MAKE MONEY FAST $$$" for example). On UNIX systems, a 'killfile' is such a set of criteria used to remove unwanted articles, and the term 'killfile' is still applied to filtering functions on some news clients.
Articles in comp.sys.mac.comm, showing articles which have been hilited by filters for the words 'NewsWatcher', 'Netscape' and 'Eudora' in article subjects.
How filters work Filters simply scan through the text of article headers (which may be headers containing the author's name, organization, the article subject line, the date of the article, the number of lines in the article or several other headers) looking for certain text strings or regular expressions. Articles that are found can then either be hilited (causing the author or title to be colored in the subject window, as above), or killed (meaning that they are marked as read before you ever see them). For more fine control over article priorities, you can have filters assign a score to articles which they match, and set up a collection of filters with a range of scores corresponding to your level of interest in articles which they match. Scoring is described in more detail below.

For each group, or hierarchy of groups on the news system, you can also specify whether articles that do not match the filters you have set up are kept or deleted. This makes it possible to set up a collection of filters that catch only the articles that you are interested in, and delete the rest. This is described in more detail below.

FILTERS INTERFACE

The Filters Window

 The list of filters which are currently defined is shown in the Filters Window, which is accessible from the Windows menu (just below the 'Full Group List' item). This window provides you with complete control over you filter collection, through extensive Drag and Drop support, and two dialogs that appear when items are double-clicked.
The Filters Window, which displays a hierarchical list of filters, clustered by the group category to which they apply. The icons show information about the filters, such as whether they expire (small watch on the icon), or whether they cannot be overriden by other filters (checkbox on the icon).
Drag and Drop in the Filters window Most filter operations can be achieved through Drag and Drop. For example, you can drag an group into this window from a list of subscribed groups, and a new group category will be created for it (if it does not already exist).

The results from a drag from the Full Groups window depend on what item was dragged. You can drag a folder item (for a level of the group hierarchy) to create a holder for filters which apply to all groups which belong to that part of the hierarchy. For example, dragging the comp.sys.mac folder item will create a filter category comp.sys.mac.*, filters within which will apply to all groups in this tree, such as comp.sys.mac.comm and comp.sys.mac.programmer.codewarrior. Thus, the total set of filters which are applied to articles in a group are the filters specific to that group, as well as those defined for all higher levels in the hierarchy.

Dragging into the Filters Window from a window displaying a list of postings (referred to as the Subject window) creates one or more new filters for the dragged article(s). If you clicked on the author name to start the drag, these will be created as author filters, or if you dragged from the subject column, they will be subject filters by default. The dotted outline that you see when dragging tells you how the filters will be created. Of course, after creating the filters you can double-click on them in the Filters Window and change the settings.

When a window showing one article is foremost (and when an article is hilited in a window showing the list of articles for a group), a couple of entries on the 'Filter' submenu of the Edit menu are enabled. These are 'Filter this author...' and 'Filter this subject...', which create author or subject filters for the active articles respectively.

You can drag filters around in this list to move them to different groups, or re-order them within the group (which changes the order in which they are applied to articles). Dragging a group item within this list simply moves all its filters. You can also drag items to and from the desktop, to create clippings files, which are used for filter conversion.

Double-clicking a filter item in the Filters Window will bring up the Filter Edit dialog, described in more detail below. Double-clicking on a filter collection item (with the small folder icon) brings up the Group Filter Edit dialog, also described below.

CREATING FILTERS
For detailed instructions on creating different types of filters, see the page on creating and editing filters.

FILTER MISCELLANY
Setting filter defaults Each time you create a filter, it is initialized according to some defaults (and, if appropriate, according to the action that created it). Those defaults you can set from the Preferences dialog. Go to the 'Filter options' panel, and click the button entitled 'Set filter defaults'. In the resulting dialog, you can specify those defaults. Be aware that there are two independent color schemes that you can set, for kill filters, and for keep/hilight filters. Move between them by choosing kill or keep, hilight from the action popup menu.
Filters and download time Some filters require additional information to be downloaded from the news server to work, which also depends on your preferences for displaying authors, lines and dates, and whether you are using the XOVER command to get headers from the server. The rather complex interactions between the various options are described in the document on downloading time.
The Filters File

 By default, the filters file, where the filters are stored, is created in the same folder as the MT-NewsWatcher Preferences file, and is called 'MT-NewsWatcher Filters'. But since MT-NewsWatcher stores an alias to this file, rather than hard-coding the path, you can move the filters file wherever you like on your disk, and the program should find it when it starts up. If it cannot, or it finds it in the Trash, you will get a warning. If you have more than one filters file (after, say, duplicating the one that was created), you can specify which one to use in the 'Filtering options' Preferences panel.

It's probably a good idea to keep a spare copy of the filter file, when you have built up a good collection of filters. Accidents do happen, and filter files do get damaged. Sometimes. You could also just select all the filters and drag them to the desktop to make a text clipping (which encodes filter information) as a backup. To restore, just drag this clipping back into the filters window.

Converting YA-NewsWatcher filters Converting filters from YA-NewsWatcher requires Brian Clark's Filter-to-TEXT utility. Run this on your filters file, to get a text file with the filter specifications in it. The text file looks something like: 
...
#      
GROUP=.
KEPT=kFalse
#
STRING=money
HEADER=subject
MATCH=kContainsTheString
EXPIRE=0
IGNORECASE=kTrue
SCORE=0
LABEL=0
...
Now, select this file in the Finder, and drag it into MT-NewsWatcher's Filters window. MT-NewsWatcher will open the file, parse its dragged text, and create a set of filters as close as it can to the YA-NW filters.

Because of differences in the filter format of the two programs, the conversion is not always perfect; in particular, lines and dates filters will not convert well if you used both 'longer' and 'shorter', or 'before' and 'after' numbers in the same YA-NW filter. They will also be created with the default hilite colors (as set in preferences).

To convert MT-NewsWatcher filters back to YA-NW, drag them out of the filters window into the Finder, to create a text clipping, or directly to a drag-aware editor. Then save into a TEXT file, and use Brian's TEXT-to-Filter utility to convert back to YA-NW filters.

Suggest some filters! You will soon build up quite a collection of filters as you realise how useful they are. From my own completely biased, Mac-based viewpoint, here are some that I use:
  • *
    • Kill if 'subject' contains 'money'
    • Kill if 'subject' contains '$$$'
    • Hilite if 'author' contains 'smfr@santafe.edu'
  • comp.sys.mac.*
    • Hilite if subject contains 'NewsWatcher'
    • Hilite if 'author' contains 'apple.com'
and so on. You can see more in the snapshot of my filters window, near the top of this page, and there are some funky regular expression-based filters on the page on regular expressions.

Because MT-NewsWatcher can save filters in Clippings files, you can swap them with other people (you need to send them as encoded attachments by email). If you have a really funky regular expression-based filter, feel free to email me a copy. Alternatively, you can simply drag and drop into a text-capable program (e.g. directly into Eudora) to exchange them in text format. You loose the colors, but the rest comes through fine.

Download filters Scott Forbes is maintaining a page from which you can download MT-NewsWatcher filters, in clipping format.
MT-NewsWatcher
Download Basics Speech Recognition Filtering Cool Features