Help with carriage return in regular expression

[cosmo]

Hello all,

  I want to generate a regular expression that considers a carriage return as part of the pattern.  After RegExpr_FindPatternInText() started returning success--but no match--to all of the patterns that I could think of, I started to dig into regexpr.c.  It appears that a carriage return is treated as a multi-line delimiter in TextToSearch and that it is eliminated from the actual match logic.  What!?!?  Only characters (on any single line) that precede a \n or \r are considered in the match.

  It is insightful to make the match logic smart enough to span lines, but it results in a misbehavior when a user specifically wants the carriage return to be considered.  Unless I am completely off the mark, this is a fundamental flaw in NI's RegEx library.

Orlan

[MJF]

Fundamental flaw? I'm not so sure about that. The regular expression functions appear to be using a subset of the Posix RE syntax which by default considers a new line or carriage return as a line delimiter. I have not used the CVI functions much (because they used to be unsafe for multi-threading) so I do not know if this default behavior can be overridden. From the help file in CVI 8.0, it does not appear that the behavior can be overridden.

In Perl, it is possible to tell the RE interpreter to parse \r \n characters as part of the text. If you know Perl, you could write a short script to parse the regular expression and execute it from your CVI program while passing the result in a temporary file. It's a bit clunky but it would definitely work.

[cosmo]

Hi MJF,

  The primary reason that I say 'fundamental flaw' is that if \n or \r is specified in the regular expression, then it is because I specifically want it to participate.  I would say that if I don't have any reference to \n or \r in the expression, then it is simply a convenience to handle multi-line text.  I agree that if it treads on Posix than it is perhaps questionable, but perhaps the Posix regular expression logic failed to consider this, or perhaps chose to ignore it as a situation that would happen less often.  Obviously if anyone (such as Perl guys) realized that this was missing from Posix implementations, then maybe they agree with me that it is in fact important enough to include the functionality.

Orlan

[MJF]

It may be a flaw from your perspective but it works the way regular expressions work by default in almost every implementation that I know about. The fact that Perl provides a mechanism to do multiline regular expression searches is an exception to the general implementation. But that is neither here nor there, the important thing to focus on is figuring out how to do what it is you want to do.

One brute-force suggestion is to use the Scan and Fmt functions to replace the \r and \n characters in the string with another character prior to running your regular expression check. Another would be to use strtok() to break it up into substrings based on the position of the new lines.

These may not be the most elegant ways to do it, but either of these approaches will likely let you achieve what you want to do.

[cosmo]

The discussion is good.  After more digging, it appears that the consideration of <newline> chars was an issue with IEEE 1003.1-2001.  It continues to cause some heartburn with IEEE 1003.1-2004.  Unfortunately, in the Shell & Utilities volume there was supposed to be a list of utilities with a clear indication of whether they permitted matching of <newline> chars.  Obviously CVI is unlikely to be a part of this volume.

After studying regexpr.c, it is very clear that matching is done line-at-a-time and <newline> chars are rendered useless.  It would be nice for NI to clarify this in their help files (panels, online help files, etc.) that the \x?? syntax matches most hex syntax--not all--because of the way that this function was written to ignore parsing of <newline> chars.

Martin, I saw your post for CVI/PERL support but I will be able to tokenize my strings along <newline> chars now that I understand this.  I appreciate it.  I would like to see some consideration in future NI CVI regex libraries to consider the text pointed to as an inline char stream.  I can't, at present, see where the multi-line style offers any advantages over an inline style, and it will be even more accurate than the existing Posix minimum requirements (which by the way don't specify whether multi-line support must be included).

Orlan

[menchar]

Yup - this sure fooled me.    Tried to match crg rtn and it don't work.

NI doesn't support the \s expression either (any whitespace) which would be a help - most regexp implementations seem to have \s and \S.

Menchar

[menchar]

After spending some time with the NI regexpr implementation, I have to say it could be better.

Providing a mode to allow parsing a newline (or allowing a newline as a metacharacter in an expression) may not be default operation on many implementations but most implementations do provide a mode that will parse newlines.  Perl does, as noted, so does GNU Emacs, java.util.regex, Ruby, .NET, tcl, GNU awk, flex, and Python.

I suppose we could say it's better than nothing. 

Menchar

[Mohan]

Yes, the CVI regular expression functions are quite basic and by no means full-featured. We will probably update this some time in the future. If your program calling into .NET assemblies is not an issue then you may want to look at the example in samples\dotnet\RegEx - this shows how to use the .NET regular expression engine from CVI. The .NET regex stuff may not be fully compliant with open standards but is definitely full-featured and is documented at: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpgenref/html/cpconRegularExpressio...

Best regards