CLc@sCddkZddkZddkTddkTdddYZdS(iN(t*tScancBsjeZdZdZedZeeiZdZ d%d%dZ dZ dZ dZdZd Zd%d Zd Zd Zd ZdZdZdZdZdZdZdZdZdZdZdZdZ dZ!dZ"dZ#dZ$dZ%dZ&d Z'd%d!Z(d"Z)d#Z*d$Z+RS(&s Stream scanner object. Written by John W. Shipman (john@nmt.edu), New Mexico Tech Computer Center, Socorro, NM 87801. This class is based on an original in the Icon language that I've used heavily. Its methods include a number that are built-ins in Icon but had to be added here. Uses the Log object for all error logging (which see). Exported methods: Scan ( fileName, commentPrefix=None, callback=None ) .close() # Close this object .atEndLine() # True iff self is at end of line .error ( text,... ) # Log a scan error message .warning ( text,... ) # Log a scan warning .msgKind ( kind, text, ... ) # Log a generic message .msgCount ( kind=None ) # Return count of messages .nextLine() # Read next line, if there is one .message ( text, ... ) # Log a message with our prefix .write ( text, ...) # Log a message without our prefix .move ( n ) # Advance position in current line .tab ( n ) # Move to position in current line .isPos ( n ) # Is current position n? .find ( s ) # Find string s on current line .upto ( c ) # Find first character not in a Cset .deblank ( ) # Skip whitespace on many lines .deblankLine ( ) # Skip whitespace on current line .any ( c ) # Is next char in a given Cset? .tabAny ( c ) # Advance if next char is in a Cset .many ( c ) # Find leading characters in a Cset .tabMany ( c ) # Advanced past leading chars in a Cset .match ( s ) # Does the line start with s? .matchArb ( s ) # Like match(), but case-insensitive .tabMatch ( s ) # Advance if line starts with s .tabMatchArb ( s ) # Like tabMatch(), but case-insensitive .reMatch ( r ) # Match regular expression .tabReMatch ( r ) # Like tabMatch(), but uses regular expr. .moveMatch ( m ) # Advance using a MatchObject (from `re') .integer ( maxLen=None ) # Parser for an integer .fixed ( ) # Parser for a fixed-point number .flatCset ( n, c ) # Parse fixed-sized field matching Cset .flatInt ( n ) # Parse fixed-sized integer field Exported elements: .atEndFile # True iff self is at end of file .lineNo # Current line number of self, counting from 1 .pos # Position in current line, must be in the range # [0,len(line)]. READ-ONLY!! .line # Same as self.rawLine, except comments (if any) # are removed. READ-ONLY!! State and invariants: .fileName If a string was passed to the constructor's fileName argument, that string is stored here, otherwise it is None. .file File handle for reading the input file. INV: Must be readable. .commentPrefix As passed to the constructor; string or None. .callback As passed to the constructor; a function or None. .lineNo Line number of self.line, counting from 1. .rawLine The current line from the input, as a string; may be empty; never has a trailing newline; valid only if .atEndFile is false, else "". .echoed 1 if a scan error has been logged against the current line of self, else 0. .atEndFile 1 if the input has been exhausted, else 0. tErrors s*** cCst|to||_t||_nd|_||_||_||_d|_d|_ d|_ d|_ d|_ d|_ |idS(s [ if (fileName is a string or file) and (commentPrefix is a string or None) and (callBack is a function or None) -> if (fileName names a readable file) -> return a new Scan object with (commentChar) as the comment character (if given) and using (callback) as the callback procedure (if given) and positioned at the beginning of the first line (if there is one) if (fileName is a readable file handle) -> return a new Scan object with (commentChar) as the comment character (if given) and using (callback) as the callback procedure (if given), reading from fileName at its current position else -> raise IOError ] itN(t isinstancetstrtfileNametopentfiletNonet commentPrefixtcallbacktlineNotrawLinetlinetpostechoedt atEndFiletnextLine(tselfRR R ((s!/u/john/projects/pystyler/scan.pyt__init__Xs            cCs(|io|iid|_ndS(sv [ self := self, closed; all further operations on self are undefined ] N(RtcloseR (R((s!/u/john/projects/pystyler/scan.pyRs  cCs)|it|ijodSndSdS(s~ [ if self is at end of line -> return 1 else -> return 0 ] iiN(RtlenR(R((s!/u/john/projects/pystyler/scan.pyt atEndLinescGs)ti|d}|i|i|dS(s [ Log() +:= an error message showing the current line and position of self, with all (text) arguments used as the message ] RN(tstringtjointmsgKindtERROR(RtLttext((s!/u/john/projects/pystyler/scan.pyterrorscGs&ti|d}|it|dS(s [ Log() +:= an error message showing the current line and position of self, with all (text) arguments used as the message ] RN(RRRt WARNING_KIND(RRR((s!/u/john/projects/pystyler/scan.pytwarningscGs|ipd|_|iod|i|if}nd|i}|iod||i|f}ntid|d|inti|d}tid|i d ti ||d S( s [ Log() +:= a message of kind (kind) showing the current line and position of self, with all (text) arguments used as the message ] isFile `%s', line %dsLine %ds%s [%s]s --- s Rt t^N( RRR R tLogtwriteR RRRR(RtkindRtwhereR((s!/u/john/projects/pystyler/scan.pyRs      cCstid|S(s Returns the count of errors or other kinds of messages. [ if kind is None -> return number of error messages in Log() else if kind is a string -> return number of messages of kind (kind) in Log() ] R%(R#tcount(RR%((s!/u/john/projects/pystyler/scan.pytmsgCountscCs|iodSn|ii|_t|idjo!d|_d|_|_dSn|iddjo|id |_n|i|_d|_|id|_d|_|i o@t i |i|i }|djo|id|!|_qndS(s [ if self is at end of file or on the last line -> return 0 else -> self := self moved to the beginning of the next line return 1 ] iiRis ( RRtreadlineR RRRR RR Rtfind(Rt commentPos((s!/u/john/projects/pystyler/scan.pyRs$       cGs*|iti|d}|i|dS(s [ Log() +:= a line consisting of self.MESSAGE_PREFIX with all (text) arguments concatenated to it ] RN(tMESSAGE_PREFIXRRR$(RRR((s!/u/john/projects/pystyler/scan.pytmessage9scGs&ti|d}ti|dS(sw [ Log() +:= a line consisting of all (text) arguments, concatenated ] RN(RRR#R$(RRR((s!/u/john/projects/pystyler/scan.pyR$DscCs|i|}d|jot|ijnp tn||ijo|}|i}n|i}|}||_|i||!}|S(s [ if n is a nonnegative integer -> if (current position)+(n) is somewhere on the current line -> self := self advanced by n characters return the string between the current position and (current position)+(n), possibly empty else -> raise IndexError ] i(RRRt IndexError(RtntnewPostloPosthiPostresult((s!/u/john/projects/pystyler/scan.pytmoveOs '    cCs|i|}d|jot|ijnp tn||ijo|}|i}n|i}|}||_|i||!}|S(s [ if newPos is an integer -> if (newPos < 0) and (position (len(self.line)-newPos-1) is somewhere on the current line) -> self.pos := len(self.line)-newPos-1 return the string between the current position and position (len(self.line)-newPos-1) (possibly empty) else if position newPos is somewhere on the current line -> self.pos := newPos return the string between the current position and position newPos (possibly an empty string) else -> raise IndexError ] i(t _Scan__effPosRRR.R(RR0teffPosR1R2R3((s!/u/john/projects/pystyler/scan.pyttab{s'    cCs.|djot|id|Sn|SdS(sJConverts a negative position to the equivalent positive position. iiN(RR(Rtp((s!/u/john/projects/pystyler/scan.pyt__effPoss cCs/|i|}|i|jo|SndSdS(s- Checks to see if the current line is at position p. Supports end-relative indexing: use -1 to check if the string is at end of line, -2 to see if it is one before end of line, and so on. Returns the (nonnegative) position if true, else returns None. [ if (p<0) and (self.pos==(len(self.line)-p-1) -> return len(self.line)-p-1 else if (newPos >=0) and (self.pos=p) -> return p else -> return None ] N(R5RR (RR8R6((s!/u/john/projects/pystyler/scan.pytisPosscCs8ti|i||i}|djodSn|SdS(s [ if string s occurs on the remainder of the current line -> return the position of the first occurrence's first character else -> return None ] iN(RR*RRR (RtsR8((s!/u/john/projects/pystyler/scan.pyR*s cCsIxBt|it|iD]%}|i|i|o|SqqWdS(sB [ if c is a Cset -> if there is a character found in c on the current line at or after the current position -> return the position of the first such character (counting from 0) else -> return None ] N(trangeRRRthasR (Rtcti((s!/u/john/projects/pystyler/scan.pytuptos  cCsEx>|ip3|i|i|io|iqdSqWdS(s [ self := self advanced past all spaces or tabs, skipping lines if necessary until reaching a character that is not space or tab, or end of file ] N(RttabManyt WHITE_CSETRR(R((s!/u/john/projects/pystyler/scan.pytdeblanks    cCs|i|idS(s [ self := self advanced past all spaces or tabs, until reaching a character that is not space or tab, or end of line, whichever comes first ] N(RARB(R((s!/u/john/projects/pystyler/scan.pyt deblankLine0scCsF|iodSn|i|i|io|idSndSdS(s [ if c is a Cset -> if the current line of self starts with a character found in c -> return the position past that character else -> return None ] iN(RR R=RR(RR>((s!/u/john/projects/pystyler/scan.pytany<s  cCs1|i|}|djodSn|i|S(s: [ if c is a cset -> if the current line of self starts with a character found in c -> self := self advanced by one character return the (old!) first character of self else -> return None ] N(RER R7(RR>R?((s!/u/john/projects/pystyler/scan.pyttabAny\s cCsj|i|djodSn |i}x<|t|ijo%|i|i|o|d}q*W|S(sU [ if c is a Cset -> if the current line of self starts with one or more characters found in c -> return the position of the first character that is not in c, or end of line, whichever comes first else -> return None ] iN(RER RRRR=(RR>R?((s!/u/john/projects/pystyler/scan.pytmanys cCs5|i|}|djodSn|i|SdS(sL [ if c is a cset -> if the current line of self starts with one or more characters found in c -> self := self advanced past all such characters return the matching characters from self else -> return None ] N(RGR R7(RR>R?((s!/u/john/projects/pystyler/scan.pyRAs cCso|it|t|ijodSn||i|i|it|!jo|it|SndSdS(s [ if s is a string -> if the current line of self starts with s -> return the position after the matching part else -> return None ] N(RRRR (RR;((s!/u/john/projects/pystyler/scan.pytmatchs #'cCs|it|t|ijodSnti|}ti|i|i|it|!}||jo|it|SndSdS(s [ if s is a string -> if the current line of self starts with s, ignoring case -> return the position just past the matching part else -> return None ] N(RRRR Rtupper(RR;tsupperttupper((s!/u/john/projects/pystyler/scan.pytmatchArbs#) cCs5|i|}|djodSn|i|SdS(s [ if s is a string -> if the current line of self starts with s -> self := self advanced past that return s else -> return None ] N(RHR R7(RR;R?((s!/u/john/projects/pystyler/scan.pyttabMatchs cCs5|i|}|djodSn|i|SdS(s" [ if s is a string -> if the current line of self starts with s, ignoring case -> self := self advanced past that return the part that matched else -> return None ] N(RLR R7(RR;R?((s!/u/john/projects/pystyler/scan.pyt tabMatchArbs cCs^d}t|tijo&|i|i}ti||}n|i|i|i}|S(siRegular expression version of .match() [ if (r is a string or a regular expression object as returned by re.compile()) -> if (r is a string) and (r, interpreted as a regular expression, matches the input -> return the MatchObject from the match if (r is a compiled regular expression) -> and (r matches the input -> return the MatchObject from the match else -> return None ] NOTE: To advance the position past a MatchObject m, use scan.move(m.end()) N(R ttypettypest StringTypeRRtreRH(RtrRttargettm((s!/u/john/projects/pystyler/scan.pytreMatch@s  cCs9|i|}|djodSn|i||SdS(sQLike .tabMatch(), but uses a regular expression. [ if (r is a string or a regular expression object as returned by re.compile()) -> if (r is a string) and (r, interpreted as a regular expression, matches the input -> scan := scan advanced past the matching part return the MatchObject for the match if (r is a compiled regular expression) -> and (r matches the input -> return the MatchObject for the match else -> return None ] N(RVR t moveMatch(RRSRU((s!/u/john/projects/pystyler/scan.pyt tabReMatch~s   cCs!|i|i|idS(sAdvance past the string represented by a MatchObject m [ if (m is a MatchObject) -> self := self advanced by the length of the match from m ] N(R4tendtstart(RRU((s!/u/john/projects/pystyler/scan.pyRWscCs|i|i}|djodSn|i|}|dj o=t||jo*|idt|d|d|!}nti|S(s [ if (maxLen is an integer or None) -> if (maxLen) is &null, or the line in self starts with a digit but not more than (maxLen) of them -> self := self advanced past all leading digits return those digits, as integer type else if (maxLen) is given and the line in self starts with more than (maxLen) digits -> self := self advanced past all leading digits Log() +:= error, integer too long return the first (maxLen) digits from self else -> return None ] s)This integer is too long, the maximum is t.iN( RAt DIGIT_CSETR R7RRRRtatoi(RtmaxLenR?R((s!/u/john/projects/pystyler/scan.pytintegers   cCs|id}|p d}nd}|i|i}|o||}n|djodSn|id}|dj o;|d}|i|i}|dj o||}qn|S(sJ [ let have the syntax [ "-" ] ... [ "." [ ... ] ] in: if self starts with a -> self := self advanced past the return the else -> return None ] t-RR[N(RMR RAR\(RtsignRtmantissaR?ttrail((s!/u/john/projects/pystyler/scan.pytfixeds       cCsE|i|}|djp||i|jodSn|i|S(s/ [ if n is a positive integer and c is a Cset -> if self begins with n characters found in c -> self := self advanced by n characters return the next n characters from self else -> return None ] N(RGR RR4(RR/R>R?((s!/u/john/projects/pystyler/scan.pytflatCset$s  cCse|i|t|ijo|i|i|i|!}ndSd}d}x ||djo|d}qNW||djo|d}d}nx ||djo|d}qW|ii||oTd}xO|t|jo3|ii||o|||}|d}qWndS|t|jo%|i||_|ti|SndSdS(s [ if n is a positive integer -> if self starts with an integer of size n, left-padded with blanks, and with a possible "-" sign located anywhere within those blanks -> self := self advanced by n characters return that signed integer as integer type else -> return None ] iiR!R`iRN(RRRR R\R=RR](RR/RRatofft magnitude((s!/u/john/projects/pystyler/scan.pytflatIntCs.   N(,t__name__t __module__t__doc__RtCsetRBRtdigitsR\R,R RRRRR RR(RR-R$R4R7R5R:R*R@RCRDRERFRGRARHRLRMRNRVRXRWR_RdReRh(((s!/u/john/projects/pystyler/scan.pyR sLC =  ( < , 1    0 # # #  ! ! > ' 7 > ((RPRtcsettlogR(((s!/u/john/projects/pystyler/scan.pyss