summaryrefslogtreecommitdiffstats
path: root/tests
Commit message (Collapse)AuthorAgeFilesLines
* hash: new functions hash-seq and hash-isecp.Kaz Kylheku2025-06-131-0/+7
| | | | | | | | | | | * hash.c (hash_seq, hash_isecp): New functions. (hash_init): hash-seq and hash-isecp intrinsics registered. * hash.h (hash_seq, hash_isecp): Declared. * tests/010/hash.tl: New tests. * txr.1: DoOcumented.
* *-get-buf: bug in skipping non-seekable streams.Kaz Kylheku2025-06-021-0/+6
| | | | | | | | | | | | | | * stdlib/getput.tl (sys:get-buf-common); Fix incorrect algorithm for skipping forward in a stream that doesn't support seek-stream. The problem is that when the seek amont is greater than 4096, it does nothing but 4096 byte reads, which will overshoot the target position if it isn't divisible by 4096. The last read must be adjusted to the remaining seek amount. * tests/018/getput.tl: New test case using property-based approach to show that the read-based skip in get-buf-common fetches the same data as the seek-based skip.
* json: *read-bad-json* allows single quoted strings.Kaz Kylheku2025-06-021-0/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It happens in the wild that sometimes JSON-like data must be processed in which strings are delimited by single quotes rather than double quotes. The data is valid Javascript syntax, so JS people don't even notice anything is wrong. * parser.c (struct parser): New member, json_quote_char. This helps the scanner keep track of which closing character it is expecting. * parser.c (parser_common_init): Initialize json_quote_char. * parser.l (JPUNC, NJPUNC): Include single quote (ASCII apostrophe) in JPUNC, and exclude it from NJPUNC. (grammar): When we see either a double quote or single quote in JLIT mode, we return it as itself if that character is the delimiter for the currently scanned string. Otherwise we return it as a LITCHAR, which gets accumulated by the parser into the current string. Include the double. When we see either a double quote or single quote, we transition to the JLIT state. The parser will check whether a single quoted literal is allowed. We allow \' escapes in a single-quote literal unconditionally. We allow them in a double-quoted literal also, but only in read bad JSON mode. * parser.y (json_val): Recognize single-quoted literals, but generate an error unless in read bad JSON mode. Also, error production for unterminated single quote only diagnosed that way in read bad JSON mode, otherwise rejected as invalid JSON. * tests/010/json.tl: New tests. * txr.1: Documented. * lex.yy.c.shipped, y.tab.c.shipped: Regenerated.
* buf: stream: switch approach for unget_char.Kaz Kylheku2025-05-291-0/+17
| | | | | | | | | | | | | | | | | | | We switch to the method used in string streams for ungetting characters, whereby we UTF-8 encode the pushed back character and push back the subsequent bytes, thereby unifying character and byte pushback. * buf.c (struct buf_strm): Remove member unget_c. (buf_strm_mark): Remove reference to unget_c. (strm_get_char): Remove code for obtaining previously pushed back character from s->unget_c stack. (buf_strm_unget_char): Rewrite using the approach of using utf8_encode to write the multi-byte representation of the character into utf8_tiny_buf, and then pushing back the bytes. (make_buf_stream): Don't initialize removed unget_c. * tests/018/streams.tl: New tests.
* buf: implement fill_buf operation for buf streams.Kaz Kylheku2025-05-281-0/+13
| | | | | | | * buf.c (buf_strm_fill_buf): New function. (buf_strm_ops): Wire in buf_strm_fill_buf operation. * tests/018/streams.tl: New tests.
* streams: seek operation for string byte input stream.Kaz Kylheku2025-05-281-1/+14
| | | | | | | * stream.c (byte_in_seek): New function. (byte_in_ops): Wire in byte_in_seek. * tests/018/streams.tl: New tests.
* buf: fix seek and truncate operations.Kaz Kylheku2025-05-271-0/+29
| | | | | | | | | | | | | | | | | | | | * buf.c (buf_strm_seek): fix incorrect from-end calculation which must be an addition, not subtraction. Throw a file-error, not generic error. When the seek size exceeds the buffer size, extend it with zeros. (buf_strm_truncate): Completely revised. When the requested length lies beyond the current position, the buffer's length is set to that position, which may truncate or extend it. When the requested length lies below the current position, the buffer is truncated only to the current position, not below. The byes below the position, down to the truncation position, are obliterated to zero. Added missing check for negative offset. * tests/018/streams.tl: New tests. * txr.1: Documentation added.
* streams: seek and truncate ops for string streams.Kaz Kylheku2025-05-251-0/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch makes seek-stream and truncate-stream work for string output streams, and seek-stream for string input streams. * stream.c (string_in_seek): New static function. (string_in_ops): Wire seek operation to string_in_seek. (struct string out): New member, len. Keeps track of the length of the string, so that fill can be freely positioned. (string_out_put_string): We can no longer add the string with a null terminator, because the put operation could be happening at any position. We only add the null terminator when we are writing the data at the end. This function also now supports buffer extension: the seek operation can seek beyond the current string. The seek operation then calls string_out_put_string with a null string. This function then grows th buffer as needed. In that case there is a need to fill the space with space characters. (string_out_truncate, string_out_seek): New static functions. (string_out_ops): Wire in string_out_seek and string_out_truncate. (make_string_output_stream); Initialize new so->len member to zero. (get_string_from_stream_common): New function, renamed from get_string_from_stream, and taking a parameter to optionally request non-destructive readout. (stream_init): Update registration of get-string-from-stream to get_string_from_stream_common. * stream.h (get_string_from_stream_common): Declared. (get_string_from_stream): Becomes inline function which calls get_string_from_stream, defaulting the argument. Why I didn't add the argument to get_string_from_stream is not to have to edit numerous calls to get_string_to_stream throughout the code base. * tests/018/streams.tl: New tests. * txr.1: Documentation updated to correct text claiming that string streams don't support truncate-stream and seek-stream, and describe the support in detal.
* streams: replace get_line virtual with new interface.Kaz Kylheku2025-05-241-0/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stream.h (struct strm_ops): The simple get_line virtual is replaced by get_string, which takes a character limit and a delimiting stop character. (strm_ops_init): Rename get_line parameter to get_string. (get_string_s): Declared. (generic_get_line): Declaration removed. (generic_get_string, get_delimited_string): Declared. * stream.c (get_string_s): New symbol variable. (unimpl_get_line): Function removed. (unimpl_get_string): New function. (null_get_line): Function removed. (null_get_string): New function. (fill_stream_ops): Configure ops->get_string rather than ops->get_line. (null_ops): Wire null_get_string in place of null_get_line. (generic_get_line): Renamed to generic_get_string. (generic_get_string): Implement the limit and stop_char parameters. (get_line_limited_check): New static function. (stdio_ops): Wire in generic_get_string instead of generic_get_line. (tail_get_line): Replaced by tail_get_string. (tail_get_string): Call generic_get_string instead of generic_get_line, and pass the limit and stop_char arguments down. (tail_ops): Wire in tail_get_string instead of tail_get_line. (pipe_ops): Wire generic_get_string instead of generic_get_line. (dir_get_line): Renamed to dir_get_string. (dir_get_string): Use get_line_limited_check to defend against unhandled argument values. (dir_ops): Wire dir_get_string instead of dir_get_line. (string_in_get_line): Replaced by string_in_get_string. (string_in_get_string): Implement limit and stop_char parameters. (string_in_ops): Wire string_in_get_string instead of string_in_get_line. (strlist_in_get_line): Replaced with strlist_in_get_string. (strlist_in_get_string): Use get_line_limited_check to defend against unsupported arguments. (strlist_in_ops): Wire in strlist_in_get_string instead of strlist_in_get_line. (cat_get_line): Replaced by cat_get_string. (cat_get_string): Rather than recursing into the get_line public interface, we fetch the stream's get_string virtual and pass all arguments to it. (cat_stream_ops): Wire cat_get_string instead of cat_get_line. (record_adapter_get_line): Replaced by record_adapter_get_string. (record_adapter_get_string): use get_line_limited_check to guard against unsupported arguments. (record_adapter_ops): Wire record_adapter_get_string instead of record_adapter_get_line. (get_line): Implement using get_string virtual now. We pass UINT_PTR_MAX as limit, which means no character limit, and '\n' as the delimiter for reading a line. (get_delimited_string): New function, which exposes the full semantics of the get_string virtual. (stream_init): Initialize get_string_s. Register get-delimited-string function. Use get_string_s symbol in registration of get-string. * strudel.c (strudel_get_line): Replaced by strudel_get_string. (strudel_get_string): Call look up the get-string method and pass all arguments to it, encoded into Lisp values in the right way, nil indicating not present. (strudel_ops): Wire strudel_get_string in place of strudel_get_line. * parser.c (shadow_ops_template): Replace generic_get_line with generic_get_string. * buf.c (buf_strm_ops): Likewise. * socket.c (dgram_strm_ops): Likewise. * gzio.c (gzio_ops_rd): Likewise. * stdlib/stream-wrap.tl (stream-wrap get-line): Method replaced by (stream-wrap get-string). This calls get-delimited-string rather than get-line. * tests/018/streams.tl: New tests, mainly concerned with the new logic in the string input stream which has its own implementation of get_string with several cases. * txr.1: Document new get-delimited-string function, and the get-string method of the delegate stream, removing the documentation for removed get-line method.
* parser: json: bugfix: nuke lookahead token.Kaz Kylheku2025-05-241-0/+7
| | | | | | | | | | The get-json function leaves a lookahead token in the parser, which is interpreted by the next call to the parser. * parser.c (prime_parser_post); Obliterate the yy_char token in the JSON case, just like we do for iread. * tests/018/streams.tl: New test case.
* parser: recover stream data buffered by lexer.Kaz Kylheku2025-05-231-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After parsing out of a stream, we attach a shadow stream which temporarily patches the operations to make it appear that bytes that were taken into the lexer have been pushed back. This lets us call an ordinary input operation after a parsing operation to read the data immediately following the parsed-out construct. (Almost: there is still the issue of the parser consuming one token of lookahead in some situations.) * stream.h (struct strm_base): New member shadow_obj. This is a context pointer used by the shadow stream operations. (generic_fill_buf): Declare previously internal function. * stream.c (strm_base_init): Initialize shadow_obj to null. (generic_fill_buf): Function changed to external linkage. Also, reloads the ops pointer from the stream on each loop iteration. This is because it can change; part of the buffer may be filled by shadow_get_byte, which can detach the shadow operations, so then the rest of the buffer is filled by something else like stdio_get_byte. (generic_get_line): Reload ops in in the loop, like in gneric_fill_buf, for the same reason. * parser.l: Include <stddef.h> for ptrdiff_t. (scanner_has_buffered_bytes, scanner_get_buffered_bytes): New functions. * parser.c (SHADOW_TAB_SIZE): New preprocessor symbol. (shadow_tab): New static array. (struct shadow_context, struct shadow_ungetch): New struct types. (lisp_parse_impl): After calling parse, call parse_shadow_stream_attach to attach the shadow stream context and operations onto the stream. (shadow_detach, shadow_destroy_op, shadow_mark_op, shadow_put_string, shadow_put_char, shadow_put_byte, shadow_get_char_callback, shadow_get_char, shadow_unget_char_callback, shadow_unget_char, shadow_get_byte, shadow_unget_byte, shadow_put_buf, shadow_close, shadow_flush, shadow_seek, shadow_truncate): New static functions. (shadow_ops_template): New static structure. (customize_shad_ops): New static function. (parser_shadow_stream_attach): New function. (parser_free_all): New function. * parser.h (scanner_has_buffered_bytes, scanner_get_buffered_bytes, parser_shadow_stream_attach, parser_free_all): Declared. * txr.c (free_all): Call parser_free_all. * tests/018/streams.tl: New test cases. * lex.yy.c.shipped: Regenerated.
* streams: get-char for string byte input streams.Kaz Kylheku2025-05-221-0/+13
| | | | | | | | | | | | | | | | | | | | | | | String byte input streams extended to provide characer input (get-char), and any mixture of unget-byte and unget-char. Also fill-buf is supported. * stream.c (struct byte_input): New ut8_decoder_t member ud. (struct byte_input_ungetch): New struct type. (byte_in_get_char_callback, byte_in_get_char, byte_in_unget_char_callback, byte_in_unget_char): New functions. (byte_in_ops): Wire in byte_in_get_char, byte_in_unget_char and byte_in_unget_byte. Also generic_fill_buf. (make_string_byte_input_stream): Initialize the UTF-8 decoder. * tests/018/streams.tl: New tests. * txr.1: Documented.
* streams: improve pushback semantics of stdio streams.Kaz Kylheku2025-05-221-0/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * utf8.[ch] (utf8_getc, utf8_ungetc): New functions which allow the push-back buffer of the decoder to be accessed. We can use the decoder's push-back buffer to implement a stream's byte push-back, so that the behavior is then consistent: invalid bytes pushed back by the decoder are treated uniformly with bytes pushed back using unget-char. * stream.c (stdio_switch): Bugfix: reset the UTF8 decoder when changing direction. Without this, it is possible that pushed back bytes in the decoder's buffer will be read, even though write operations moved the position. Thus stdio_switch is now defined as a function regardless of whether CONFIG_STDIO_STRICT is in effect. (stdio_get_byte): If there are pushed back characters present, throw an error. Otherwise, try to get a byte from the UTF8 buffer's pushback first via utf8_getc. If that produces something, just return it. Otherwise fall back on reading from the stdio stream. (stdio_unget_byte): If there are pushed back characters present, throw an error. Otherwise push back the character using utf8_ungetc. If that reports no space, throw an error. (stdio_fill_buf): Take bytes from the push-back buffer int he UTF8 decoder first, then fread the rest from the stdio stream, if necessary.
* expander: handle format string in quasi.Kaz Kylheku2025-05-191-0/+6
| | | | | | | | * eval.c (expand_quasi): Quasiliteral sys:var items now have a format string, which we have to propagate to the expansion. * tests/012/quasi.tl: New tests.
* quasistrings: support format notation.Kaz Kylheku2025-05-181-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds support to quasiliterals to have the inserted items formatted via a format conversion specifier, for example @~3,3a:abc is @abc modified by ~3,3a format conversion. When the inserted value is a list, the conversion is distributed over the elements individually. Otherwise it applies to the entire item. * eval.c (fmt_tostring, fmt_cat): Take additional format string argument. If it isn't nil, then do the string conversion via the fmt1 function rather than tostring. (do_format_field): Take format string argument, and pass down to fmt_cat. (format_field); Take format string argument and pass down to do_format_field. (fmt_simple, fmt_flex): Pass nil format string argument to fmt_tostring. (fmt_simple_fmstr, fmt_flex_fmstr): New static functions, like fmt_simple and fmt_flex but with format string arg. Used as run-time support for compiler-generated quasilit code for cases when format conversion specifier is present. (subst_vars): Extract the new format string frome each variable item. Pass it down to fmt_tostring, format_field and fmt_cat. (eval_init): Register sys:fmt-simple-fmstr and sys:flex-fmstr intrinsics. * eval.h (format_field): Declaration updated. * lib.c (out_quasi_str_sym): Take format string argument. If it is present, output it after the @, followed by a colon, to reproduce the read notation. (out_quasi_str): Pass down the format string, taken from the fourth element of a sys:var item of the quasiliteral. For simple symbolic items, pass down nil. * match.c (tx_subst_vars): Pass nil as new argument of format_field. The output variables of the TXR Pattern language do not exhibit this feature. * parser.l (FMT): New pattern for matching the format string part. (grammar): The rule which recognizes @ in quasiliterals optionally scans the format notation, and turns it into a string attached to the token's semantic value, which is now of type val (see parser.y remarks). * parser.y (tokens): The '@' token's %type changed from lineno to val so it can carry the format string. (q_var): If format string is present in the @ symbol, then include it as the fourth element of the sys:var form. This rule handles braced items. (meta): We can no longer get the line number from the @ item, so we get it from n_expr. (quasi_item): Similar to q_var change here. This handles @ followed by unbraced items: symbols and other expressions. * stdlib/compiler.tl (expand-quasi-mods): Take format string argument. When the format string is present, then generate code which uses the new alternative run-time support functions, and passes them the format string as an argument. (expand-quasi-args): Extend the sys:var match to extract the format string if it is present. Pass it down to expand-quasi-mods. * stdlib/match.tl (expand-quasi-match): Add an error case diagnosing the situation when the program tries to use a format-conversion-endowed item in a quasilit pattern. * stream.[ch] (fmt1): New function. * tests/012/quasi.tl: New tests. * txr.1: Documented. * lex.yy.c.shipped, y.tab.c.shipped: Regenerated.
* infix: add missing ~ complement operator.Kaz Kylheku2025-05-161-0/+4
| | | | | | | | | * stdlib/infix.tl (toplevel): New ~ operator, prefix at level 35, tied to lognot function. * tests/012/infix.tl: New tests. * txr.1: Documented.
* buf: bug: empty fill pattern.Kaz Kylheku2025-05-161-0/+2
| | | | | | | | | | * buf.c (prepare_pattern): Do not return a zero length pattern for an empty buffer. Callers don't deal with this properly. Return a one byte pattern that is zero. * test/012/buf.tl: Test case added. * txr.1: Documented.
* buf: make-buf and buf-set-length take pattern.Kaz Kylheku2025-05-161-0/+31
| | | | | | | | | | | | | The init-val parameter of make-buf and buf-set-length is generalized to a possibly multi-byte fill pattern. * buf.c (make_buf, buf_do_set_len): Change init-val parameter to init-pat, and implement. We optimize the to memset when it's a one-byte pattern. * tests/012/buf.tl: New tests. * txr.1: Documented.
* syntax: allow for multiline qref chaining.Kaz Kylheku2025-05-151-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | Introducing a relaxation in the obj.slot.(method arg) syntax. There can be whitespace to the right of the dot, for splitting across multiple lines, as: obj. slot. (method arg) * parser.l (OREFDOT): Allow optional whitespace to the right of .? * parser.y (n_expr): Add a n_expr LAMBDOT n_expr phrase, with same semantic rule as n_expr '.' n_expr. We cannot add optional whitespace after . in the lexer because that is ambiguous with LAMBDOT. * tests/012/syntax.tl: New tewt cases. * txr.1: Documented. * lex.yy.c.shipped, y.tab.c.shipped: Regenerated.
* New string compression/decompression functions.Kaz Kylheku2025-05-151-1/+6
| | | | | | | | | | | | * buf.c (str_compress, str_decompress): New functions. (buf_init): str-compress, str-decompress intrinsics registered. * lib.[ch] (string_utf8_from_buf): New function. * tests/012/buf.tl: New tests. * txr.1: Documented.
* New function: buf-xor-pattern.Kaz Kylheku2025-05-111-0/+11
| | | | | | | | | | | | | | * buf.c (buf_xor_pattern): New function. (buf_init): Register buf-xor-pattern intrinsic. * buf.h (buf_xor_pattern): Declared. * lib.[ch] (unsup_obj): Change function to external linkage and declare in header. * tests/012/buf.tl: New tests. * txr.1: Documented.
* match: new macros in the "each" family.Kaz Kylheku2025-05-081-0/+34
| | | | | | | | | | | | | | | | | * autoload.c (match_set_entries): Trigger autoload on new symbols in function namespace: each-match-case, collect-match-cases, append-match-cases, keep-match-cases, each-match-case-product, collect-match-case-products, append-match-case-products, keep-match-case-products. * stdlib/match.tl (each-match-case, collect-match-cases, append-match-cases, keep-match-cases, each-match-case-product, collect-match-case-products, append-match-case-products, keep-match-case-products): New macros. * tests/011/patmatch.tl: New tests. * txr.1: Documented.
* infix: bug: non-infix expressions conflated with infix.Kaz Kylheku2025-05-071-13/+13
| | | | | | | | | | | | | | | | | | | The problem is that (parse-infix '(x < y < z)) and (parse-infix '(x < (< y z)) produce exactly the same parse and will be treated the same way. But we would like (< y z) to be left alone. The fix is to annotate all compound terms such that finish-infix will not recurse into them. * stdlib/infix.tl (parse-infix): When an operand is seen that is a compound expression X it is turned into @X, in other words (sys:expr X). (finish-infix): Recognize (sys:expr X) and convert it into X without recursing into it. * tests/012/infix.tl: Update a number of test cases. * txr.1: Documented.
* New function: buf-binary-width.Kaz Kylheku2025-05-071-0/+56
| | | | | | | | | | | * buf.c (buf_binary_width): New function. (buf_init): Register buf-binary-width intrinsic. * buf.h (buf_binary_width): Declared. * tests/012/buf.tl: New tests. * txr.1: Documented.
* New function: buf-count-ones.Kaz Kylheku2025-05-061-0/+140
| | | | | | | | | | | * buf.c (buf_count_ones): New function. (buf_init): Register buf-count-ones intrinsic. * buf.h (buf_count_ones): Declared. * tests/012/buf.h: New tests. * txr.1: Documented.
* New function: buf-test.Kaz Kylheku2025-05-061-0/+24
| | | | | | | | | | | * buf.c (buf_test): New function. (buf_init): Register buf-test intrinsic. * buf.h (buf_test): Declared. * tests/012/buf.tl: New tests. * txr.1: Documented.
* New function: buf-zero-p.Kaz Kylheku2025-05-061-0/+22
| | | | | | | | | | | * buf.c (buf_zero_p): New function. (buf_init): Register buf-zero-p intrinsic. * buf.h (buf_zero_p): Declared. * tests/012/buf.tl: New tests. * txr.1: Documented.
* New function: buf-bit.Kaz Kylheku2025-05-051-0/+6
| | | | | | | | | | | * buf.c (buf_bit): New function. (buf_init): Register buf_bit intrinsic. * buf.h (buf_bit): Declared. * tests/012/buf.h: New tests. * txr.1: Documented.
* New function: buf-bitset.Kaz Kylheku2025-05-051-0/+16
| | | | | | | | | | | * buf.c (buf_bitset): New function. (buf_init): Register buf-bitset intrinsic. * buf.h (buf_bitset): Declared. * tests/012/buf.tl: New tests. * txr.1: Documented.
* New functions: buf-not and buf-trunc.Kaz Kylheku2025-05-051-0/+27
| | | | | | | | | | | * buf.c (buf_not, buf_trunc): New functions. (buf_init): Register buf-not and buf-trunc intrinsics. * buf.h (buf_not, buf_trunc): Declared. * tests/012/buf.h: New tests. * txr.1: Documented.
* New functions: buf-and, buf-or, buf-xor.Kaz Kylheku2025-05-051-0/+69
| | | | | | | | | | | * buf.c (buf_and, buf_or, buf_xor): New functions. (buf_init): buf-and, buf-or and buf-xor intrinsics registered. * buf.h (buf_and, buf_or, buf_xor): Declared. * tests/012/buf.tl: New tests. * txr.1: Documented.
* New function: buf-fash.Kaz Kylheku2025-05-051-0/+63
| | | | | | | | | | | * buf.c (buf_fash): New function. (buf_init): buf-fash intrinsic registered. * buf.h (buf_fash): Declared. * tests/012/buf.tl: New tests. * txr.1: Documented.
* New function; buf-ash.Kaz Kylheku2025-05-051-0/+90
| | | | | | | | | | | | | * buf.c (err_oflow): New static function. (buf_compress): Use err_oflow. (buf_ash): New function. (buf_init): Register buf-ash intrinsic. * buf.h (buf_ash): Declared. * tests/012/buf.tl: New tests. * txr.1: Documented.
* buf: buf-put-buf: wrong argument order.Kaz Kylheku2025-05-031-0/+25
| | | | | | | | | | | | | | | * buf.c (buf_put_buf): Fix incorrect argument order, contrary to documentation and inconsistent with other buf-put-* functions. (compat_buf_put_buf): New static function. Detects wrong argument order and swaps. (buf_init): Register buf-put-buf intrinsic to new static function. Thus buf-put-buf conforms to the documentation, but also works if called with the incorect old argument order. * test/012/buf.tl: New tests. * txr.1: Documentation added to clarify behaviors when put operation is out of the range of the destination buffer.
* New function: progf.Kaz Kylheku2025-05-021-0/+5
| | | | | | | | | * eval.c (do_progf, progf): New static functions. (eva_init): Register progf intrinsic. * tests/012/callable.tl: New test. * txr.1: Documented.
* infix: use superfix in fft test case.Kaz Kylheku2025-05-021-1/+1
| | | | | * tests/012/infix.tl (fft): express loop condition more succinctly as a relational compound.
* infix: superfix relational operators; better code.Kaz Kylheku2025-05-011-0/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit extends infix with a post-processing step applied to the output of parse-infix which improves the code, and also implements math-like semantics for relational operators, which I'm calling superfix. Improving the code means that expressions like a + b + c, which turn into (+ (+ a b) c) are cleaned up into (+ a b c). This is done for all n-ary operators. superfix means that clumps of certain operators behave as a compound. For instance a < b <= c means (and (< a b) (<= b c)), where b is evaluated only once. Some relational operators are n-ary; for those we generate the n-ary expression, so that a = b = c < d becomes (and (= a b c) (< c d)). * stdlib/infix.tl (*ifx-env*): New special variable. We use this for communicating the macro environment down into the new finish-infix function, without having to pass a parameter through all the recursion. (eq, eql, equal, neq, neql, nequal, /=, <, >, <=, >=, less, greater, lequal, gequal): These operators become right associative, and are merged into a single precedence level. (finish-infix): New function which coalesces compounds of n-ary operations and converts the postfix chains of relational operators into the correct translation of superfix semantics. (infix-expand-hook): Call finish-infix on the output of parse-infix, taking care to bind the *ifx-env* variable to the environment we are given. * tests/012/infix.tl: New tests. * txr.1: Documented.
* infix: test cases for prefix phony.Kaz Kylheku2025-04-231-2/+19
| | | | | | | | | | * test/012/infix.tl: New tests. (*compiling*) New variable. (unless-compiling): New macro. One :error test elicits a warning during the compiled version of the test that is hard to squelch, so we just remove it from the compiled test case. (fft): Remove unused variable i; all for loops locally bind i.
* infix: revise auto-detection algorithm.Kaz Kylheku2025-04-211-6/+6
| | | | | | | | | | | | | | | * stdlib/infix.tl (detect-infix): Redesign. New algorithm looks only at the first two or three elements. Arguments that are not operators are only considered operands if they don't have function bindings. This is important because sometimes the logic is applied to the arguments in a DWIM bracket form, like [apply / args], which we don't want to treat as (/ apply args). * tests/012/infix.tl: New test. * txr.1: Redocumented.
* infix: more test cases.Kaz Kylheku2025-04-182-0/+60
| | | | | | | | * tests/012/infix.tl: New tests providing some coverage of ifx, and its phony infix also. Big test case with FFT function. * tests/common.tl (msstest): New macro.
* New: HMAC functions.Kaz Kylheku2025-04-161-0/+8
| | | | | | | | | | | | * autoload.c (hmac_set_entries, hmac_instantiate): New static functions. (autoload_init): Register autoload of hmac module. * stdlib/hmac.tl: New file. * tests/013/chksum.tl: New tests. * txr.1: Documented.
* chksum: support stream arguments in more places.Kaz Kylheku2025-04-151-2/+7
| | | | | | | | | | | | | | | | | | | | | | * chksum.c (sha1_stream_read, sha256_stream_read, md5_stream_read): New static function. (sha1_stream_impl, sha256_stream_impl, md5_stream_impl): Logic moved into new functions. (sha1, sha256, md5): Support stream argument, via corresponding stream_impl function, passing nil as the size to snarf the whole stream. This could have been implemented without the above refactoring. In other words, the _stream functions only need to be used now when a limit on the number of bytes must be specified. (sha1_hash, sha256_hash, md5_hash): Support a stream argument. * gencksum.txr: All above chksum.c changes actually generated from here. * tests/013/chksum.tl: New tests. * txr.1: Documentation updated.
* infix: revise auto-detection and parsing.Kaz Kylheku2025-04-091-9/+9
| | | | | | | | | | | | | | * stdlib/infix.tl (parse-infix): Change how we treat fn (arg ...) and elem [arg ...] forms. These now translate to (fn (arg ...)) and [elem (arg ...)] rather than (fn arg ...) and [elem arg ...]. (detect-infix): detect certain op [arg ...] forms as infix. (infix-expand-hook): Revise detection logic to handle bracket expression forms, and parenthesized single terms. The latter are needed to reduce [elem (atom)] to [elem atom]. * tests/012/infix.tl: Fix up some tests. * txr.1: Documented.
* infix: add quadratic-roots testKaz Kylheku2025-04-061-0/+10
| | | | | | | * tests/012/infix.tl (quadractic-roots): New function. Add couple of tests. * txr.1: Add quadratic-roots as example to ifx macro.
* infix: add tests.Kaz Kylheku2025-04-062-1/+68
| | | | | | * tests/012/infix.tl: New file. * tests/012/compile.tl: Add infix to compiled tests.
* expand-hook-combine: bugfix.Kaz Kylheku2025-04-011-5/+34
| | | | | | | | | | | | | | | | | | | | | | | | | Here we fix bugs in expand-hook-combine, imrprove the tests and make different recommendations in the manual about hook order. * eval.c (expand_hook_combine_fun): Fix incorrect tests which cause the next function to be ignored. * tests/011/exphook.tl: (pico-style-expand-hook): Needs tweak to evaluate constantp using standard expansion (without pico-style), so that pico-style can nest with ifx in either order. (pico-style): Now when we call expand-hook-combine we give the new hook first, and the existing one next. This behavior makes more sense as a default go-to strategy because it gives priority to the innermost hook-based macro, closest to the code. (infix-expand-hook, ifx): Add test cases which test nesting of hook-based macros. * txr.1: Opposite recommendation made about chaining of expand hooks: new first, fall back on old. Example adjusted.
* New function: expand-hook-combine.Kaz Kylheku2025-04-011-0/+17
| | | | | | | | | | | | | This function provides a functional combinator that takes the responsibility of combining expand hooks. * eval.c (expand_hook_combine_fun, expand_hook_combine): New static functions. (eval_init): Register expand-hook-combine intrinsic. * tests/011/exphook.tl: New file. * txr.1: Documented.
* match: new pattern matching macro, match-tuple-case.Kaz Kylheku2025-04-011-0/+17
| | | | | | | | | | | | | * autolod.c (match_set_entries): Autoload match module on match-tuple-case. * match.tl (match-tuple-case): New macro. * tests/011/patmatch.tl: New tests. The macro is trivial; if lambda-match works, the macro works. * txr.1: Documented.
* New function: remove.Kaz Kylheku2025-03-271-0/+24
| | | | | | | | | | | | | | | We need a remove function that doesn't have an equality suffix, analogous to member, pos, count. * eval.c (eval_init): Register remove intrinsic. * lib.[ch] (remov): New function. Named this way to avoid clashing with the ISO C remove function in <stdlib.h>. * tests/012/seq.tl: New tests. * txr.1: Documented.
* rand: use PRNG bits more economically for small moduli.Kaz Kylheku2025-03-212-60/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The rand function always calls rand32 at least one for each call, even for small moduli that need only a few pseudo-random bits. For instance when the modulus is 2, the function requires only one pseudo-random bit from the PRNG, yet it takes 32 and throws away 31. With this commit, that changes. For moduli of 65536 or smaller, the bits are used more efficiently; and for a modulus of 2, the function can satisfy 32 calls using the bits of a single rand32_t word: one stepping of the WELL512a PRNG. * rand.c (struct rand_state): New member, shift. Holds the shift register for rand/random to take bits from, replenished from rand32() when it runs out. The shift register detects when it runs out of bits in a clever way, without any additional variable. The register is regarded as being 33 bits wide, with a top bit that is always 1. When the register is empty, a 32 bit word is taken from the PRNG. The required random bits are taken from the word, and it is then shifted to the right. (We take only power-of-two amounts out of the shift register: 1, 2, 4, 8 or 16 bits). Even the smallest shift produces enough room that the 33rd bit can be added to the word, into its shifted position. After that, the shift register is considered to have enough bits for a given modulus if its value is less than equal to the mask. I.e if we were to take bits from it, we would be including the unconditional signaling bit. At that point we clobber the shift register with a new set of 32 bits from the PRNG, take the random bits we need, shift it to the right and add the signaling bit. (opt_noshift): New static variable; indicates whether we are in compatibility mode, requiring the shift register optimization to be defeated. (make_random_state): Initialize shift register to 0 in several places. (random): Implement various small modulus cases. There are specific cases for moduli that are exactly 65536, 256, 16, 4, 3 and 2. The in-between cases are handled by shifting the bits in the same amounts as the next higher power of two from this list of sizes: 16, 8, or 4 bits. For these cases, we calculate the smallest Mersenne modulus which covers the bits of the actual moduls and use that for rejecting potential values, just as we do in the general large modulus case. For instance if the modulus is 60 (range 0 to 59), that lands into the 8 bit shift range: we pull 8 bits at a time from the shift register. But the modulus 60 is covered by the six bit mask 63. We mask each 8 bit value with 63, and if it is in the required range 0 to 59, we accept it, otherwise draw another 8 bits. (rand_compat_fixup): Initialize opt_noshift to 1 if the requested compat version is 299 or less. * tests/012/sort.tl: Fix one test case involving shuffled data. The shufle function uses rand with small moduli, so its behavior changes for the same PRNG sequence. * tests/013/maze.expected: Likewise, the generated pseudo-random maze in the maze test case is different now; we must update to the new expected output. * txr.1: Document that a value of 299 or less of the compatibility -C option has an effect on rand.
generated by cgit v1.2.3 (git 2.25.1) at 2025-07-14 23:57:15 +0000