Monday, November 28, 2005

Automated editing

Over at the American Copy Editors Society's discussion board, Merrill Perlman of the New York Times has posted a link under this headline: Something REALLY Scary

The link goes to Tansa Systems, which boasts that it is "far more effective at finding and correcting spelling, style and grammar mistakes than ordinary 'spelling checkers.'" One of its case studies (PDF) proudly notes how it has improved the fortunes of the Norwegian newspaper Bergensavisen. Of course, tucked into that case study is this sentence: "Unlike many Scandanavian newspapers, Bergensavisen has never had full-time employees dedicated solely to proofreading text." (It does, however, apparently have subeditors - the European equivalent of copy editors -- because one is quoted in the story.)

My thoughts, as posted on the ACES board:
Well, mixed emotions. If we look at our jobs in the cold light, we might say some of what we do could be done by ever-more-sophisticated automated systems. And maybe we should jettison that part of the job.

But critical questions, things like why is "blonde" relevant here or that she's "a mother of two" in a murder story, or these things would go together better if we flipped A and B, or that percentage increase doesn't make sense unless you give me the raw numbers, too, are things I don't think a computer ever will get right. So, just like the business we are in, if we peg our hats on the "commodity" stuff, we risk getting replaced. If we really concentrate on the value-added, less so.

Having said that, of course, you know darn well there are publishers who would replace large chunks of their desks in the process. I think you have to treat that as you would a charging bull elephant -- it's inevitable that it's coming your way; you just have to try to step out of the way in time.

(Also, even allowing for the European origin, I quickly found several mistakes on Tansa's Web site, so ...)

(Disclosing my bias: Were I to start a paper or news Web site today, I would look into installing an automated writing program strictly to handle processing of any press releases deemed necessary to get in but not important enough to follow up. Why have a reporter burning time doing rewrite when he or she can be out getting the goods? However, NONE of that material would ever be published without an editor's review.)

Here's Tansa's list of U.S. customers:

Magazines / Trade Journals
American Chemical Society
- Analytical Chemistry
- Chemical & Engineering News
- Environmental Science & Technology
- Modern Drug Discovery
- Today's Chemist At Work
The Chronicle of Higher Education
- The Chronicle of Higher Education
- The Chronicle of Philanthropy

Geo. J. Foster & Co.
- Foster’s Daily Democrat (Dover, N.H.)
- The Citizen and Sunday Citizen (Laconia, N.H.)
- Sanford News (Sanford, N.H.)
- Best Read Guide
- Rochester Times

The Lawton Constitution (Lawton, Okla.)
The News-Gazette (Champaign, Ill.)
Republican-American (Waterbury, Conn.)
Rockford Register Star (Rockford, Ill.)(future)
St. Cloud Times (St. Cloud, Minn.)
The Washington Times (Washington, D.C.)

Labels: , ,


At 11/29/05, 7:53 AM, Anonymous Anonymous said...

This system is not quite what people assume it to be from its advertising. We use the system. Trust me -- this will not replace people. While it's smart, it isn't THAT smart, and sometimes its suggestions are more of a pain than they are worth. It can be time-consuming to build up the dictionaries in Tansa, and it requires great diligence on the part of all staffers to do so. With people squeezed already for time, you can imagine how well we've been able to build ours. I'd chalk it up to a spellchecker on steroids, but still a spellchecker overall.

At 11/29/05, 9:53 AM, Blogger Doug said...

Thank you. That's very useful. I'll share it with the ACES list.

At 11/30/05, 1:20 AM, Anonymous Anonymous said...

For some reason I can't post on the ACES board lately. Sigh ...

Anyway, so here is an example of how it doesn't work quite as you would think.

I typed this sentence into a file and ran the program: What affect doe that have on anything, he asked?

Tansa picked up neither the effect/affect issue nor the misspelling of "does" (as doe is a real word).

The best part of the program is the ability to add your own unique proper names and such to the dictionaries. But even that sometimes doesn't quite yield the results you want. We mostly find it fixes spelling.

Stand down, everyone. We will all still have our jobs in the morning!

At 9/26/06, 1:46 AM, Anonymous Anonymous said...

are there any automated writing programs? I think they would be useful in helping teach students and citizen journalists the basics of newswriting. for example: the program asks a series of questions about say, the public meeting the reporter just attended, and from their responses generates a sample draft, which the reporter can then modify. It would provide a solid starting point and the reporter could improve on it from there.

At 9/26/06, 4:10 AM, Blogger Doug said...

Actually, that's a very good thought vis a vis citizen journalists.

There are some programs being marketed that claim to write sports stories, for instance (I see them in my local press association bulletin). One wonders how they would do with "native" input instead of an already-formatted, (supposedly) style-correct release. However, I think at least one produces copy from raw stats.


Post a Comment

<< Home