summaryrefslogtreecommitdiffstats
path: root/tests
Commit message (Collapse)AuthorAgeFilesLines
* streams: improve pushback semantics of stdio streams.Kaz Kylheku2025-05-221-0/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * utf8.[ch] (utf8_getc, utf8_ungetc): New functions which allow the push-back buffer of the decoder to be accessed. We can use the decoder's push-back buffer to implement a stream's byte push-back, so that the behavior is then consistent: invalid bytes pushed back by the decoder are treated uniformly with bytes pushed back using unget-char. * stream.c (stdio_switch): Bugfix: reset the UTF8 decoder when changing direction. Without this, it is possible that pushed back bytes in the decoder's buffer will be read, even though write operations moved the position. Thus stdio_switch is now defined as a function regardless of whether CONFIG_STDIO_STRICT is in effect. (stdio_get_byte): If there are pushed back characters present, throw an error. Otherwise, try to get a byte from the UTF8 buffer's pushback first via utf8_getc. If that produces something, just return it. Otherwise fall back on reading from the stdio stream. (stdio_unget_byte): If there are pushed back characters present, throw an error. Otherwise push back the character using utf8_ungetc. If that reports no space, throw an error. (stdio_fill_buf): Take bytes from the push-back buffer int he UTF8 decoder first, then fread the rest from the stdio stream, if necessary.
* expander: handle format string in quasi.Kaz Kylheku2025-05-191-0/+6
| | | | | | | | * eval.c (expand_quasi): Quasiliteral sys:var items now have a format string, which we have to propagate to the expansion. * tests/012/quasi.tl: New tests.
* quasistrings: support format notation.Kaz Kylheku2025-05-181-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds support to quasiliterals to have the inserted items formatted via a format conversion specifier, for example @~3,3a:abc is @abc modified by ~3,3a format conversion. When the inserted value is a list, the conversion is distributed over the elements individually. Otherwise it applies to the entire item. * eval.c (fmt_tostring, fmt_cat): Take additional format string argument. If it isn't nil, then do the string conversion via the fmt1 function rather than tostring. (do_format_field): Take format string argument, and pass down to fmt_cat. (format_field); Take format string argument and pass down to do_format_field. (fmt_simple, fmt_flex): Pass nil format string argument to fmt_tostring. (fmt_simple_fmstr, fmt_flex_fmstr): New static functions, like fmt_simple and fmt_flex but with format string arg. Used as run-time support for compiler-generated quasilit code for cases when format conversion specifier is present. (subst_vars): Extract the new format string frome each variable item. Pass it down to fmt_tostring, format_field and fmt_cat. (eval_init): Register sys:fmt-simple-fmstr and sys:flex-fmstr intrinsics. * eval.h (format_field): Declaration updated. * lib.c (out_quasi_str_sym): Take format string argument. If it is present, output it after the @, followed by a colon, to reproduce the read notation. (out_quasi_str): Pass down the format string, taken from the fourth element of a sys:var item of the quasiliteral. For simple symbolic items, pass down nil. * match.c (tx_subst_vars): Pass nil as new argument of format_field. The output variables of the TXR Pattern language do not exhibit this feature. * parser.l (FMT): New pattern for matching the format string part. (grammar): The rule which recognizes @ in quasiliterals optionally scans the format notation, and turns it into a string attached to the token's semantic value, which is now of type val (see parser.y remarks). * parser.y (tokens): The '@' token's %type changed from lineno to val so it can carry the format string. (q_var): If format string is present in the @ symbol, then include it as the fourth element of the sys:var form. This rule handles braced items. (meta): We can no longer get the line number from the @ item, so we get it from n_expr. (quasi_item): Similar to q_var change here. This handles @ followed by unbraced items: symbols and other expressions. * stdlib/compiler.tl (expand-quasi-mods): Take format string argument. When the format string is present, then generate code which uses the new alternative run-time support functions, and passes them the format string as an argument. (expand-quasi-args): Extend the sys:var match to extract the format string if it is present. Pass it down to expand-quasi-mods. * stdlib/match.tl (expand-quasi-match): Add an error case diagnosing the situation when the program tries to use a format-conversion-endowed item in a quasilit pattern. * stream.[ch] (fmt1): New function. * tests/012/quasi.tl: New tests. * txr.1: Documented. * lex.yy.c.shipped, y.tab.c.shipped: Regenerated.
* infix: add missing ~ complement operator.Kaz Kylheku2025-05-161-0/+4
| | | | | | | | | * stdlib/infix.tl (toplevel): New ~ operator, prefix at level 35, tied to lognot function. * tests/012/infix.tl: New tests. * txr.1: Documented.
* buf: bug: empty fill pattern.Kaz Kylheku2025-05-161-0/+2
| | | | | | | | | | * buf.c (prepare_pattern): Do not return a zero length pattern for an empty buffer. Callers don't deal with this properly. Return a one byte pattern that is zero. * test/012/buf.tl: Test case added. * txr.1: Documented.
* buf: make-buf and buf-set-length take pattern.Kaz Kylheku2025-05-161-0/+31
| | | | | | | | | | | | | The init-val parameter of make-buf and buf-set-length is generalized to a possibly multi-byte fill pattern. * buf.c (make_buf, buf_do_set_len): Change init-val parameter to init-pat, and implement. We optimize the to memset when it's a one-byte pattern. * tests/012/buf.tl: New tests. * txr.1: Documented.
* syntax: allow for multiline qref chaining.Kaz Kylheku2025-05-151-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | Introducing a relaxation in the obj.slot.(method arg) syntax. There can be whitespace to the right of the dot, for splitting across multiple lines, as: obj. slot. (method arg) * parser.l (OREFDOT): Allow optional whitespace to the right of .? * parser.y (n_expr): Add a n_expr LAMBDOT n_expr phrase, with same semantic rule as n_expr '.' n_expr. We cannot add optional whitespace after . in the lexer because that is ambiguous with LAMBDOT. * tests/012/syntax.tl: New tewt cases. * txr.1: Documented. * lex.yy.c.shipped, y.tab.c.shipped: Regenerated.
* New string compression/decompression functions.Kaz Kylheku2025-05-151-1/+6
| | | | | | | | | | | | * buf.c (str_compress, str_decompress): New functions. (buf_init): str-compress, str-decompress intrinsics registered. * lib.[ch] (string_utf8_from_buf): New function. * tests/012/buf.tl: New tests. * txr.1: Documented.
* New function: buf-xor-pattern.Kaz Kylheku2025-05-111-0/+11
| | | | | | | | | | | | | | * buf.c (buf_xor_pattern): New function. (buf_init): Register buf-xor-pattern intrinsic. * buf.h (buf_xor_pattern): Declared. * lib.[ch] (unsup_obj): Change function to external linkage and declare in header. * tests/012/buf.tl: New tests. * txr.1: Documented.
* match: new macros in the "each" family.Kaz Kylheku2025-05-081-0/+34
| | | | | | | | | | | | | | | | | * autoload.c (match_set_entries): Trigger autoload on new symbols in function namespace: each-match-case, collect-match-cases, append-match-cases, keep-match-cases, each-match-case-product, collect-match-case-products, append-match-case-products, keep-match-case-products. * stdlib/match.tl (each-match-case, collect-match-cases, append-match-cases, keep-match-cases, each-match-case-product, collect-match-case-products, append-match-case-products, keep-match-case-products): New macros. * tests/011/patmatch.tl: New tests. * txr.1: Documented.
* infix: bug: non-infix expressions conflated with infix.Kaz Kylheku2025-05-071-13/+13
| | | | | | | | | | | | | | | | | | | The problem is that (parse-infix '(x < y < z)) and (parse-infix '(x < (< y z)) produce exactly the same parse and will be treated the same way. But we would like (< y z) to be left alone. The fix is to annotate all compound terms such that finish-infix will not recurse into them. * stdlib/infix.tl (parse-infix): When an operand is seen that is a compound expression X it is turned into @X, in other words (sys:expr X). (finish-infix): Recognize (sys:expr X) and convert it into X without recursing into it. * tests/012/infix.tl: Update a number of test cases. * txr.1: Documented.
* New function: buf-binary-width.Kaz Kylheku2025-05-071-0/+56
| | | | | | | | | | | * buf.c (buf_binary_width): New function. (buf_init): Register buf-binary-width intrinsic. * buf.h (buf_binary_width): Declared. * tests/012/buf.tl: New tests. * txr.1: Documented.
* New function: buf-count-ones.Kaz Kylheku2025-05-061-0/+140
| | | | | | | | | | | * buf.c (buf_count_ones): New function. (buf_init): Register buf-count-ones intrinsic. * buf.h (buf_count_ones): Declared. * tests/012/buf.h: New tests. * txr.1: Documented.
* New function: buf-test.Kaz Kylheku2025-05-061-0/+24
| | | | | | | | | | | * buf.c (buf_test): New function. (buf_init): Register buf-test intrinsic. * buf.h (buf_test): Declared. * tests/012/buf.tl: New tests. * txr.1: Documented.
* New function: buf-zero-p.Kaz Kylheku2025-05-061-0/+22
| | | | | | | | | | | * buf.c (buf_zero_p): New function. (buf_init): Register buf-zero-p intrinsic. * buf.h (buf_zero_p): Declared. * tests/012/buf.tl: New tests. * txr.1: Documented.
* New function: buf-bit.Kaz Kylheku2025-05-051-0/+6
| | | | | | | | | | | * buf.c (buf_bit): New function. (buf_init): Register buf_bit intrinsic. * buf.h (buf_bit): Declared. * tests/012/buf.h: New tests. * txr.1: Documented.
* New function: buf-bitset.Kaz Kylheku2025-05-051-0/+16
| | | | | | | | | | | * buf.c (buf_bitset): New function. (buf_init): Register buf-bitset intrinsic. * buf.h (buf_bitset): Declared. * tests/012/buf.tl: New tests. * txr.1: Documented.
* New functions: buf-not and buf-trunc.Kaz Kylheku2025-05-051-0/+27
| | | | | | | | | | | * buf.c (buf_not, buf_trunc): New functions. (buf_init): Register buf-not and buf-trunc intrinsics. * buf.h (buf_not, buf_trunc): Declared. * tests/012/buf.h: New tests. * txr.1: Documented.
* New functions: buf-and, buf-or, buf-xor.Kaz Kylheku2025-05-051-0/+69
| | | | | | | | | | | * buf.c (buf_and, buf_or, buf_xor): New functions. (buf_init): buf-and, buf-or and buf-xor intrinsics registered. * buf.h (buf_and, buf_or, buf_xor): Declared. * tests/012/buf.tl: New tests. * txr.1: Documented.
* New function: buf-fash.Kaz Kylheku2025-05-051-0/+63
| | | | | | | | | | | * buf.c (buf_fash): New function. (buf_init): buf-fash intrinsic registered. * buf.h (buf_fash): Declared. * tests/012/buf.tl: New tests. * txr.1: Documented.
* New function; buf-ash.Kaz Kylheku2025-05-051-0/+90
| | | | | | | | | | | | | * buf.c (err_oflow): New static function. (buf_compress): Use err_oflow. (buf_ash): New function. (buf_init): Register buf-ash intrinsic. * buf.h (buf_ash): Declared. * tests/012/buf.tl: New tests. * txr.1: Documented.
* buf: buf-put-buf: wrong argument order.Kaz Kylheku2025-05-031-0/+25
| | | | | | | | | | | | | | | * buf.c (buf_put_buf): Fix incorrect argument order, contrary to documentation and inconsistent with other buf-put-* functions. (compat_buf_put_buf): New static function. Detects wrong argument order and swaps. (buf_init): Register buf-put-buf intrinsic to new static function. Thus buf-put-buf conforms to the documentation, but also works if called with the incorect old argument order. * test/012/buf.tl: New tests. * txr.1: Documentation added to clarify behaviors when put operation is out of the range of the destination buffer.
* New function: progf.Kaz Kylheku2025-05-021-0/+5
| | | | | | | | | * eval.c (do_progf, progf): New static functions. (eva_init): Register progf intrinsic. * tests/012/callable.tl: New test. * txr.1: Documented.
* infix: use superfix in fft test case.Kaz Kylheku2025-05-021-1/+1
| | | | | * tests/012/infix.tl (fft): express loop condition more succinctly as a relational compound.
* infix: superfix relational operators; better code.Kaz Kylheku2025-05-011-0/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit extends infix with a post-processing step applied to the output of parse-infix which improves the code, and also implements math-like semantics for relational operators, which I'm calling superfix. Improving the code means that expressions like a + b + c, which turn into (+ (+ a b) c) are cleaned up into (+ a b c). This is done for all n-ary operators. superfix means that clumps of certain operators behave as a compound. For instance a < b <= c means (and (< a b) (<= b c)), where b is evaluated only once. Some relational operators are n-ary; for those we generate the n-ary expression, so that a = b = c < d becomes (and (= a b c) (< c d)). * stdlib/infix.tl (*ifx-env*): New special variable. We use this for communicating the macro environment down into the new finish-infix function, without having to pass a parameter through all the recursion. (eq, eql, equal, neq, neql, nequal, /=, <, >, <=, >=, less, greater, lequal, gequal): These operators become right associative, and are merged into a single precedence level. (finish-infix): New function which coalesces compounds of n-ary operations and converts the postfix chains of relational operators into the correct translation of superfix semantics. (infix-expand-hook): Call finish-infix on the output of parse-infix, taking care to bind the *ifx-env* variable to the environment we are given. * tests/012/infix.tl: New tests. * txr.1: Documented.
* infix: test cases for prefix phony.Kaz Kylheku2025-04-231-2/+19
| | | | | | | | | | * test/012/infix.tl: New tests. (*compiling*) New variable. (unless-compiling): New macro. One :error test elicits a warning during the compiled version of the test that is hard to squelch, so we just remove it from the compiled test case. (fft): Remove unused variable i; all for loops locally bind i.
* infix: revise auto-detection algorithm.Kaz Kylheku2025-04-211-6/+6
| | | | | | | | | | | | | | | * stdlib/infix.tl (detect-infix): Redesign. New algorithm looks only at the first two or three elements. Arguments that are not operators are only considered operands if they don't have function bindings. This is important because sometimes the logic is applied to the arguments in a DWIM bracket form, like [apply / args], which we don't want to treat as (/ apply args). * tests/012/infix.tl: New test. * txr.1: Redocumented.
* infix: more test cases.Kaz Kylheku2025-04-182-0/+60
| | | | | | | | * tests/012/infix.tl: New tests providing some coverage of ifx, and its phony infix also. Big test case with FFT function. * tests/common.tl (msstest): New macro.
* New: HMAC functions.Kaz Kylheku2025-04-161-0/+8
| | | | | | | | | | | | * autoload.c (hmac_set_entries, hmac_instantiate): New static functions. (autoload_init): Register autoload of hmac module. * stdlib/hmac.tl: New file. * tests/013/chksum.tl: New tests. * txr.1: Documented.
* chksum: support stream arguments in more places.Kaz Kylheku2025-04-151-2/+7
| | | | | | | | | | | | | | | | | | | | | | * chksum.c (sha1_stream_read, sha256_stream_read, md5_stream_read): New static function. (sha1_stream_impl, sha256_stream_impl, md5_stream_impl): Logic moved into new functions. (sha1, sha256, md5): Support stream argument, via corresponding stream_impl function, passing nil as the size to snarf the whole stream. This could have been implemented without the above refactoring. In other words, the _stream functions only need to be used now when a limit on the number of bytes must be specified. (sha1_hash, sha256_hash, md5_hash): Support a stream argument. * gencksum.txr: All above chksum.c changes actually generated from here. * tests/013/chksum.tl: New tests. * txr.1: Documentation updated.
* infix: revise auto-detection and parsing.Kaz Kylheku2025-04-091-9/+9
| | | | | | | | | | | | | | * stdlib/infix.tl (parse-infix): Change how we treat fn (arg ...) and elem [arg ...] forms. These now translate to (fn (arg ...)) and [elem (arg ...)] rather than (fn arg ...) and [elem arg ...]. (detect-infix): detect certain op [arg ...] forms as infix. (infix-expand-hook): Revise detection logic to handle bracket expression forms, and parenthesized single terms. The latter are needed to reduce [elem (atom)] to [elem atom]. * tests/012/infix.tl: Fix up some tests. * txr.1: Documented.
* infix: add quadratic-roots testKaz Kylheku2025-04-061-0/+10
| | | | | | | * tests/012/infix.tl (quadractic-roots): New function. Add couple of tests. * txr.1: Add quadratic-roots as example to ifx macro.
* infix: add tests.Kaz Kylheku2025-04-062-1/+68
| | | | | | * tests/012/infix.tl: New file. * tests/012/compile.tl: Add infix to compiled tests.
* expand-hook-combine: bugfix.Kaz Kylheku2025-04-011-5/+34
| | | | | | | | | | | | | | | | | | | | | | | | | Here we fix bugs in expand-hook-combine, imrprove the tests and make different recommendations in the manual about hook order. * eval.c (expand_hook_combine_fun): Fix incorrect tests which cause the next function to be ignored. * tests/011/exphook.tl: (pico-style-expand-hook): Needs tweak to evaluate constantp using standard expansion (without pico-style), so that pico-style can nest with ifx in either order. (pico-style): Now when we call expand-hook-combine we give the new hook first, and the existing one next. This behavior makes more sense as a default go-to strategy because it gives priority to the innermost hook-based macro, closest to the code. (infix-expand-hook, ifx): Add test cases which test nesting of hook-based macros. * txr.1: Opposite recommendation made about chaining of expand hooks: new first, fall back on old. Example adjusted.
* New function: expand-hook-combine.Kaz Kylheku2025-04-011-0/+17
| | | | | | | | | | | | | This function provides a functional combinator that takes the responsibility of combining expand hooks. * eval.c (expand_hook_combine_fun, expand_hook_combine): New static functions. (eval_init): Register expand-hook-combine intrinsic. * tests/011/exphook.tl: New file. * txr.1: Documented.
* match: new pattern matching macro, match-tuple-case.Kaz Kylheku2025-04-011-0/+17
| | | | | | | | | | | | | * autolod.c (match_set_entries): Autoload match module on match-tuple-case. * match.tl (match-tuple-case): New macro. * tests/011/patmatch.tl: New tests. The macro is trivial; if lambda-match works, the macro works. * txr.1: Documented.
* New function: remove.Kaz Kylheku2025-03-271-0/+24
| | | | | | | | | | | | | | | We need a remove function that doesn't have an equality suffix, analogous to member, pos, count. * eval.c (eval_init): Register remove intrinsic. * lib.[ch] (remov): New function. Named this way to avoid clashing with the ISO C remove function in <stdlib.h>. * tests/012/seq.tl: New tests. * txr.1: Documented.
* rand: use PRNG bits more economically for small moduli.Kaz Kylheku2025-03-212-60/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The rand function always calls rand32 at least one for each call, even for small moduli that need only a few pseudo-random bits. For instance when the modulus is 2, the function requires only one pseudo-random bit from the PRNG, yet it takes 32 and throws away 31. With this commit, that changes. For moduli of 65536 or smaller, the bits are used more efficiently; and for a modulus of 2, the function can satisfy 32 calls using the bits of a single rand32_t word: one stepping of the WELL512a PRNG. * rand.c (struct rand_state): New member, shift. Holds the shift register for rand/random to take bits from, replenished from rand32() when it runs out. The shift register detects when it runs out of bits in a clever way, without any additional variable. The register is regarded as being 33 bits wide, with a top bit that is always 1. When the register is empty, a 32 bit word is taken from the PRNG. The required random bits are taken from the word, and it is then shifted to the right. (We take only power-of-two amounts out of the shift register: 1, 2, 4, 8 or 16 bits). Even the smallest shift produces enough room that the 33rd bit can be added to the word, into its shifted position. After that, the shift register is considered to have enough bits for a given modulus if its value is less than equal to the mask. I.e if we were to take bits from it, we would be including the unconditional signaling bit. At that point we clobber the shift register with a new set of 32 bits from the PRNG, take the random bits we need, shift it to the right and add the signaling bit. (opt_noshift): New static variable; indicates whether we are in compatibility mode, requiring the shift register optimization to be defeated. (make_random_state): Initialize shift register to 0 in several places. (random): Implement various small modulus cases. There are specific cases for moduli that are exactly 65536, 256, 16, 4, 3 and 2. The in-between cases are handled by shifting the bits in the same amounts as the next higher power of two from this list of sizes: 16, 8, or 4 bits. For these cases, we calculate the smallest Mersenne modulus which covers the bits of the actual moduls and use that for rejecting potential values, just as we do in the general large modulus case. For instance if the modulus is 60 (range 0 to 59), that lands into the 8 bit shift range: we pull 8 bits at a time from the shift register. But the modulus 60 is covered by the six bit mask 63. We mask each 8 bit value with 63, and if it is in the required range 0 to 59, we accept it, otherwise draw another 8 bits. (rand_compat_fixup): Initialize opt_noshift to 1 if the requested compat version is 299 or less. * tests/012/sort.tl: Fix one test case involving shuffled data. The shufle function uses rand with small moduli, so its behavior changes for the same PRNG sequence. * tests/013/maze.expected: Likewise, the generated pseudo-random maze in the maze test case is different now; we must update to the new expected output. * txr.1: Document that a value of 299 or less of the compatibility -C option has an effect on rand.
* New macro: letrec.Kaz Kylheku2025-03-211-0/+21
| | | | | | | | | * eval.c (me_letrec): New function. (eval_init): Register letrec intrinsic macro. * tests/012/let.tl: New file. * txr.1: Documented, and also referenced from mlet.
* Expose brace expansion bexp function.Kaz Kylheku2025-03-091-37/+37
| | | | | | | | | | | | | | | | | * autoload.c (glob_set_entries): Remove autoload on sys:brace-expand. Add usr:exp. * stdlib/glob.tl (brace-expand): Renamed to usr:bexp. (glob*): Call bexp rather than brace-expand. * tests/018/glob.tl: Rename references to sys:brace expand to bexp. * txr.1: Add section describing the bexp function. Move brace expansion documentation from glob* to this new section, adjusting the wording a little bit, mainly to avoid referring to "patterns". Point glob* documentation to bexp, which also in turn references glob*.
* glob*: add string and integer ranges to brace expansion.Kaz Kylheku2025-03-081-1/+76
| | | | | | | | | | | | | | | | * stdlb/glob.tl (bexp-parse): Recognize .. as a token. (bexp-parse-brace): If a brace expansion doesn't contain commas, then check whether it contains .. and that its elements are all strings. In that case it is a possible range expansion and we thus transform it to a (- ...) node, subject to more validation in bexp-expand. (bexp-expand): Add casees to handle range expansion, taking care that invalid forms translate to verbatim syntax. * tests/018/glob.tl: New tests. * txr.1: Documented.
* New feature: range iteration with skip.Kaz Kylheku2025-03-071-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | The notation X..Y..Z now denotes an iterable range, if X..Y is a valid iterable range on its own, and Z is a positive integer. Z gives a step size: 1 takes every element, 2 every other and so on. * lib.c (seq_iter_get_skip, set_iter_peek_skip): New static functions. (si_skip_ops): New static structure. (iter_dynamic): Function relocated earlier in file to avoid forward declaration. (seq_iter_init_with_info): When the iterated object is a range, check for the to element itself being a range. If so, it is potentially a skip iteration. Validate it and implement via a skip iterator referencing a dynamic range iterator. * lib.h (struct seq_iter): New sub-union member, ul.skip. We could use an existing member of type cnum; this is for naming clarity. * tests/012/iter.tl: New tests. * txr.1: Documented.
* New function: iterp.Kaz Kylheku2025-03-071-0/+20
| | | | | | | | | | | | | | | | | * eval.c (eval_init): Register iterp intrinsic. * lib.[ch] (iterp): New function. * tests/012/iter.tl: New tests. * txr.1: Document iterp. Update documentation for iter-more, iter-item and iter-step to more precisely identify which objects are valid arguments in terms of iterp and additional conditions, and that other objects throw a type-error exception. Fix wrong references to iter-more under documentation for iter-item. Removed obsolete text specifying that iter-step uses car on list-like sequences, a dubious behavior removed in the previous commit.
* iterator API: reject objects that don't make sense.Kaz Kylheku2025-03-061-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * lib.c (iter_more): Do not return t for unrecognized objects, but thrown an exception. Do return t for conses, which is necessary since they are iterators for lists. Also, as a special case, we return t for struct objects that don't have an iter-more method. This is needed for the documented fast protocol. Iterator objects implementing the fast protocol still get iter-more invoked on them. The client usually doesn't know that the iterator implements the fast protocol, and so calls iter-more, which unconditionally has to returns true. (iter_item): Do not fall back on car(iter) for all unhandled objects. Only conses are handled via car. All unrecognized objects trigger an exception. (iter_step): Do not try to handle list-like objects via cdr, only lists. Improve the diagnostic for hitting the end of an improper list: diagnostic shows the cons cell rather than just the terminating atom. * tests/012/iter.tl: Some test cases validating that the functions error out for strings and vectors. Much more coverage is possible here but doesn't seem worth it; e.g. that the functions reject a buffer, regex, function, ...
* awk: add CSV support.Kaz Kylheku2025-01-301-0/+18
| | | | | | | | | | | * stdlib/awk (awk-state upd-rec-to-f): Handle a new case of fs being the keyword symbol :csv, producing a field-splitting lambda that calls get-csv. * tests/015/awk-basic.tl: Several new test cases for this CSV feature. * txr.1: Documented.
* awk: use prepared lambdas for field separation.Kaz Kylheku2025-01-282-0/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Handle field separations with lambdas, similarly to record separation. The idea is that we replace the rec-to-f method, which contains a cond statement checking the variables for which field separation discipline applies, with a lambda which is updated whenever any of those ariables change. * awk.tl (awk-state): New instance slot, rec-to-f. (awk-state :postinit): Call new upd-rec-to-f method so that rec-to-f is populated with the default field separating lambda. (awk-state rec-to-f): Method removed. (awk-state upd-rec-to-f): New method, based on rec-to-f. This doesn't perform the field separation, but returns a lambda which will perform it. (awk-state loop): We must call upd-rec-to-f whenever we change par-mode, because it influences field separation. (awk-mac-let): Replace the symbol macros fs, ft, fw and kfs with new implementations that use the reactive slot mechanism provided by rslot. Whenever the awk macro assigns any of these, the upd-rec-to-f method will be called. * tests/015/awk-basic.tl: New file. These basic tests of field separation pass before and after this change. * tests/common.tl (otest, motest): New macros.
* get-csv: bugfix: return nil on EOF.Kaz Kylheku2025-01-241-1/+23
| | | | | | | | | | | | | | | | | * stream.c (get_csv): Let's add a new state init. If get_char returns nil and we are in the init state, let's bail to a nil return. While we are at it, let's not allocate the record or string until we read at least one character. If we read a character in the init state, let's allocate those two objects, and then change to the rfield state and fall through to it to handle the character. * tests/010/csv.tl: Fix one incorrect test: (tocsv "") now returns nil, as it should. Add tests for multiple record extraction, also covering missing line termination on the last record as well as CR-LF termination. * txr.1: Documented nil return conditions.
* New functions for producing CSV.Kaz Kylheku2025-01-241-0/+19
| | | | | | | | | | | | * stream.c (put_csv, tocsv): New functions. (stream_init): put-csv and tocsv intrinsics registered. * stream.h (put_csv, tocsv): Declared. * tests/010/csv.tl (mtest-pcsv): New macro. New test cases. * txr.1: Documented.
* New function: get-csv.Kaz Kylheku2025-01-211-0/+166
| | | | | | | | | | | | | * autloload.c (csv_set_entries, csv_instantiate): New static funtions. (autoload_init): Register autoload of stdlib/csv module via new functions. * stdlib/csv.tl: New file. * tests/010/csv.tl: Likewise. * txr.1: Documented.
* New macros for enumerated constants.Kaz Kylheku2025-01-201-0/+23
| | | | | | | | | | | | | * autoload.c (enum_set_entries, enum_instantiate): New static functions. (autoload_init): Register autoload of stdlib/enum module via new functions. * stdlib/enum.tl: New file. * tests/016/enum.tl: Likewise. * txr.1: Documented.
generated by cgit v1.2.3 (git 2.25.1) at 2025-07-18 22:01:42 +0000