summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
...
* infix: autodetection takes lexical functions into account.Kaz Kylheku2025-04-212-21/+24
| | | | | | | | | | * stdlib/infix.tl (funp): New macro. (detect-infix): Take environment argument and test with funp rather than fboundp. (infix-expand-hook): Pass environment to detect-infix. Also use funp in the phony infix argument test. * txr.1: Documented.
* infix: revise auto-detection algorithm.Kaz Kylheku2025-04-213-50/+95
| | | | | | | | | | | | | | | * stdlib/infix.tl (detect-infix): Redesign. New algorithm looks only at the first two or three elements. Arguments that are not operators are only considered operands if they don't have function bindings. This is important because sometimes the logic is applied to the arguments in a DWIM bracket form, like [apply / args], which we don't want to treat as (/ apply args). * tests/012/infix.tl: New test. * txr.1: Redocumented.
* infix: phony infix: try arguments as infix.Kaz Kylheku2025-04-212-7/+32
| | | | | | | | | | | | * stdlib/infix.tl (infix-expand-hook): In the phony infix logic that swaps the first two arguments, we also try the remaining arguments as a stand-alone expression, passing that through the hook. If the hook recognizes and transforms them as infix, we keep the result as one argument. Otherwise, we just take the original arguments. I already committed some test cases for this which are failing. * txr.1: Documented.
* op: reduce and mitigate multiple expansion.Kaz Kylheku2025-04-191-4/+6
| | | | | | | | | * stdlib/op.tl (sys:op-expand): Bind *expand-hook* to nil in several places so that the unavoidable multiple expansions we perform do not re-invoke hooks. Finally, when we interpolate the calculated lambda-interior into the output templates, we mark it noexpand since the material already underwent several expansions.
* expander: noexpand mechanism.Kaz Kylheku2025-04-192-1/+70
| | | | | | | | | | | | | * eval.c (noexpand_s): New symbol variable, holding sys:noexpand symbol. (do_expand): When a (sys:noexpand X) form is seen, then produce the expansion X without recursing into it; in other words, sys:noexpand is like quote, but against expansion. (noexpand): Static function. (eval_init): Initialize noexpand_s. Register noexpand as intrinsic function. * txr.1: Document noexpand function.
* infix: more test cases.Kaz Kylheku2025-04-182-0/+60
| | | | | | | | * tests/012/infix.tl: New tests providing some coverage of ifx, and its phony infix also. Big test case with FFT function. * tests/common.tl (msstest): New macro.
* autoload: reset expand hook while loading.Kaz Kylheku2025-04-171-0/+1
| | | | | | | | | * autoload.c (autload_try): Just like we set *package* and *package-alist* to sane values before loading, we must also reset *expand-hook* to nil. We would not want autoloads to be processed with some custome expansion that breaks them. This matters in the rare case when they are being loaded from .tl sources rather than .tlo.
* listener: auto infix mode.Kaz Kylheku2025-04-172-7/+50
| | | | | | | | | | * parser.c (listener_auto_infix_s): New symbol variable for *listener-auto-infix-s* special. (repl): After potentially transforming the input line via auto compound mode, check for auto infix mode. If turned on, wrap the expression in ifx. (parse_init): Initialize listener_auto_infix_s. Register the special variable named by the symbol.
* Rebind *expand-hook* in load and compile-file.Kaz Kylheku2025-04-174-4/+41
| | | | | | | | | | | | * eval.c (loadv): Rebind *expand-hook* to its current value, like we do with *package*. * match.c (v_load): Likewise. * stdlib/compiler.tl (compile-file-conditionally): Likewise. * txr.1: Documented.
* New: HMAC functions.Kaz Kylheku2025-04-164-0/+137
| | | | | | | | | | | | * autoload.c (hmac_set_entries, hmac_instantiate): New static functions. (autoload_init): Register autoload of hmac module. * stdlib/hmac.tl: New file. * tests/013/chksum.tl: New tests. * txr.1: Documented.
* chksum: support stream arguments in more places.Kaz Kylheku2025-04-154-39/+136
| | | | | | | | | | | | | | | | | | | | | | * chksum.c (sha1_stream_read, sha256_stream_read, md5_stream_read): New static function. (sha1_stream_impl, sha256_stream_impl, md5_stream_impl): Logic moved into new functions. (sha1, sha256, md5): Support stream argument, via corresponding stream_impl function, passing nil as the size to snarf the whole stream. This could have been implemented without the above refactoring. In other words, the _stream functions only need to be used now when a limit on the number of bytes must be specified. (sha1_hash, sha256_hash, md5_hash): Support a stream argument. * gencksum.txr: All above chksum.c changes actually generated from here. * tests/013/chksum.tl: New tests. * txr.1: Documentation updated.
* special-operator-p: don't report error entries as t.Kaz Kylheku2025-04-142-2/+46
| | | | | | | | | | | * eval.c (special_operator_p): Do not return t for certain symbols that are wired to error-throwing functions in the table, whose purpose it is to detect certain unexpanded syntax that is handled internally by the expander. * txr.1: Documentation udpated in several places to clarify that certain operators are not special operators. Also expander-let is reclassified from Macro to Ooerator.
* expander-let: rewrite in C.Kaz Kylheku2025-04-143-62/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The expander-let mechanism wants to be more tightly integrated into the expander than a macro. The reason is that the macro makes an explicit call to the expander. But the expansion it returns will be processed again by the expander. In the tightly integrated expander-let, we can avoid this; the expander can assume that the function which processes expander-let completely processes it and consequently can be tail called. All in all, since the expander is written in C, a utility which is this close to the heart of the expander should be implemented together with it in C. * eval.c (expander_let_s): New symbol variable. (expand_expander_let): New function. (do_expand): Tail call expand_expander_let when encountering a form headed by the expander-let symbol. (eval_init): Initialize expander_let_s variable with interned symbol. Also wire it into the special op table as an error entry, similarly to macrolet and a few others. * autoload.c (expander_let_set_entries, expander_let_instantiate): Static functions removed. (autoload_init): Autoload registration for expander-let module removed. * stdlib/expander-let.tl: File removed. Also, it should be noted that the the expander-let macro in this file has a a tiny bug: it refers to a sys:dv-bind symbol which should have been sys:dvbind. That means it is evaluating the (sys:dvbind ...) forms, which means it binds special variables twice: once in that evaluation, and then again in progv.
* doc: infix: document precedence demotion rule.Kaz Kylheku2025-04-141-0/+145
| | | | | | | * txr.1: Add a subsection to Infix Syntax titled Precedence Demotion Rule which describes our infix parser's enhancement to operator precedence parsing, possibly not found anywhere else.
* infix: revise auto-detection and parsing.Kaz Kylheku2025-04-093-42/+203
| | | | | | | | | | | | | | * stdlib/infix.tl (parse-infix): Change how we treat fn (arg ...) and elem [arg ...] forms. These now translate to (fn (arg ...)) and [elem (arg ...)] rather than (fn arg ...) and [elem arg ...]. (detect-infix): detect certain op [arg ...] forms as infix. (infix-expand-hook): Revise detection logic to handle bracket expression forms, and parenthesized single terms. The latter are needed to reduce [elem (atom)] to [elem atom]. * tests/012/infix.tl: Fix up some tests. * txr.1: Documented.
* expand-hook: implement in macroexpand-1.Kaz Kylheku2025-04-081-8/+35
| | | | | | | | | | | | | | | | | | | When expanding a form with macroexpand-1 or macroexpand, and an expand hook is in effect, we must call the hook. Otherwise macrology that relies on macroexpand or macroexpand-1 might not work with expander-driven syntax. I'm running into not being able to, for instance, use the swap macro on a pair of infix expressions because of this. macroexpand-1 must behave like the real macro expander. eval.c (do_macroexpand_1): After looking up whether the form is a macro, pass it through the hook if it is defined. If the hook transforms the code, then iterate on this. We pass the form through the expand hook even if it is not a macro call, and classify it based on whether is a function call, dwim brackets form or other, careful not to pass down special operators.
* expand-hook: process DWIM brackets forms.Kaz Kylheku2025-04-082-12/+27
| | | | | | | | | | | It turns out we need this for infix. * eval.c (do_expand): Call the expand hook for forms headed by the dwim operator, passing that operator symbol as the type indicator. * txr.1: Document this and also that the expand hook doesn't receive symbol macros.
* expand-hook: don't process symbol macros.Kaz Kylheku2025-04-081-10/+0
| | | | | | | | | | There isn't an obvious use for calling the expander hook for symbolic forms that are macros. (Especially while not calling it for symbolic forms that are not macros.) * eval.c (do_expand): Do not call the expand hook for symbol macro forms.
* infix: new operators, revise precedence.Kaz Kylheku2025-04-083-29/+148
| | | | | | | | | | | | | | | | * autoload.c (infix_set_entries): Intern new symbols ||, &&, !, !=, &=, |=, &&=, ||=, >>=, <<=, ~=, %=, *=, %=, <<, >>, &, |, ~, % and //. * stdlib/infix.tl: revise precedence of calculating assignment operators. Add shifts, bitwise operators, modulo, C-like synonyms for some operators, numerous new calculating assignments. (sys:mod-set, sys:and-set, sys:or-set, sys:logand-set, sys:logxor-set, sys:logior-set, sys:ash-set, sys:asr-set, sys:asr): New macros to provide the implementation of operation combinations that will only be available via infix.
* New macros: mul and div.Kaz Kylheku2025-04-083-1/+33
| | | | | | | | | * autoload.c (place_set_entries): Add mul and div symbols as autoload triggers. * stdlib/place.tl (mul, div): New macros. * txr.1: Documented.
* bident/lident: allow | character.Kaz Kylheku2025-04-086-4126/+4119
| | | | | | | | | | | | | | | | We allow the | character to be an identifier or constituent of identifers. This is going to be immediately useful in infix syntax to provide the C-like operators. * parser.l (BSCHR, NSCHR, ID_END): Add | character. * genvim.txr (glyph, iskeyword): Add | character. * lex.yy.c.shipped, * tl.vim, * txr.vim: Regenerated. * txr.1: Documented.
* place: move set-mask, get-mask out of defset.Kaz Kylheku2025-04-073-9/+9
| | | | | | | | | | * autoload.c (place_set_entries, defset_set_entries): Move the set-mask and clear-mask symbols from def_set_entries to place_set_entries. * stdlib/defset.tl (set-mask, clear-mask): Functions removed from here. * stdlib/place.tl (set-mask, clear-mask): Functions moved here.
* infix: add quadratic-roots testKaz Kylheku2025-04-062-0/+24
| | | | | | | * tests/012/infix.tl (quadractic-roots): New function. Add couple of tests. * txr.1: Add quadratic-roots as example to ifx macro.
* doc: mention whitespace issue in infixKaz Kylheku2025-04-061-0/+13
| | | | | | * txr.1: Make it clear that whitespace is required between atoms in infix expressions most of the time, so certain compact notations from C-like syntax are not achieved.
* infix: document.Kaz Kylheku2025-04-061-0/+355
| | | | * txr.1: New major section Infix Syntax.
* infix: remove unnecessary rewrite ruleKaz Kylheku2025-04-061-2/+0
| | | | | | | * stdlib/infix.tl (parse-infix): We don't need to recognize consecutive [...][...], because the rule which reduces any element followed by [...] does the job.
* infix: add tests.Kaz Kylheku2025-04-062-1/+68
| | | | | | * tests/012/infix.tl: New file. * tests/012/compile.tl: Add infix to compiled tests.
* infix: revise auto-detection.Kaz Kylheku2025-04-051-17/+20
| | | | | | | | | | | * stdlib/infix.tl (parse-infix): Drop usr: package prefix; autoload.c interns this symbol in the usr package. (detect-infix): New function, whose single responsibility is determining whether the argument expression should be treated via parse-infix. (infix-expand-hook): Simplified by using detect-infix function.
* infix: whitespace fix.Kaz Kylheku2025-04-051-1/+1
| | | | | * stdlib/infix.tl (infix-error): Remove trailing whitespace.
* infix: define = operator mapping to identityKaz Kylheku2025-04-051-0/+2
| | | | | | | | * stdlib/infix.tl (toplevel): New prefix operator = at 0 precedence. This is useful for specifying an infix formula that is not being autodetected by ifx nicely. For instance an expression containing only array references can be obtained as (= a[i][j]).
* infix: dynamic precedence algorithmKaz Kylheku2025-04-041-3/+8
| | | | | | | | | | | | | | | | | | | | | We implement a dynamic precedence algorithm whereby when an infix operator is immediately followed by a clump of one or more consecutive prefix operators, the infix operator's precedence is lowered to one less than the lowest one of the prefix operators. This creates nice handling for situations like (sqrt x + y - sqrt z + w) whose visual symmetry parses into (- (sqrt (+ x y)) (sqrt (+ z w))) rather than subordinating the second sqrt to the first one. * stdlib/infix.tl (parse-infix): Before processing an infix operator, calculate the prefix of the rest of the input that consists of nothing but consecutive prefix operators, and if it is nonempty, then use it to adjust the effective precedence used for the infix operator. This algorithm must only ever lower the precedence, never raise it.
* infix: assignment must be right associativeKaz Kylheku2025-04-041-1/+1
| | | | | | * stdlib/infix.tl (toplevel): The := operator must be assoc :right so a := b := c becomes (set a (set b c)) and not (set (set a b) c).
* infix: adjust operator expected diagnostic.Kaz Kylheku2025-04-041-1/+1
| | | | | | | | | * stdlib/infix.tl (parse-infix): The operator expected diagnostic can occur not just before an an operand, but before an prefix operator. For instance "a cos b". An operator is expected between a and cos. We don't want to say "before operand cos" because cos is an operator.
* Initial implementation of infix expressions.Kaz Kylheku2025-04-032-0/+196
| | | | | | | | | | | | | | | | The infix module provides a macro called ifx. Forms (evaluated expressions) enclosed inside ifx at any nesting level, which are not special operator or macro forms, are subject to automatic detection of an infix notation, which is transformed into regular Lisp. The notation is based on Lisp atoms; no read syntax is introduced. Infix may be freely mixed with ordinary Lisp. * autoload.c (infix_set_entries, infix_instantiate): New static functions. (autoload_init): Register new infix module for autoload. * stdlib/infix.tl: New file.
* expander: expand arguments after hook processing.Kaz Kylheku2025-04-021-14/+16
| | | | | | | | | | | | * eval.c (do_expand): Do not expand the arguments of a function call prior to checking for and dispatching the expand hook. The expand hook may rewrite the entire form and the arguments, so that those expansions are then thrown away. Not only is it wasteful to calculate them but possibly wrong. A form that is rewritten by a hook may have strange syntax, such that the hook itself will get confused if it is unleashed recursively on the constituent fragments of the syntax.
* signal: consider SIGSYS as synchronous signal.Kaz Kylheku2025-04-011-3/+3
| | | | | | | | | * signal.c (is_cpu_exception): Function renamed to is_synchronous, and also checks for SIGSYS. SIGSYS isn't a CPU exception, but is exception-like in that the program is immediately informed that it did something wrong. (sig_handler): Follow above rename.
* expand-hook-combine: bugfix.Kaz Kylheku2025-04-013-15/+45
| | | | | | | | | | | | | | | | | | | | | | | | | Here we fix bugs in expand-hook-combine, imrprove the tests and make different recommendations in the manual about hook order. * eval.c (expand_hook_combine_fun): Fix incorrect tests which cause the next function to be ignored. * tests/011/exphook.tl: (pico-style-expand-hook): Needs tweak to evaluate constantp using standard expansion (without pico-style), so that pico-style can nest with ifx in either order. (pico-style): Now when we call expand-hook-combine we give the new hook first, and the existing one next. This behavior makes more sense as a default go-to strategy because it gives priority to the innermost hook-based macro, closest to the code. (infix-expand-hook, ifx): Add test cases which test nesting of hook-based macros. * txr.1: Opposite recommendation made about chaining of expand hooks: new first, fall back on old. Example adjusted.
* constantp: muffle all expander warnings.Kaz Kylheku2025-04-011-2/+13
| | | | | | | | | | | | | | This fixes nuisance diagnostics from constantp, such as when invalid forms are given. An example is (constantp '(1)). * eval.c (no_ub_warn_expand): New name for previous no_warn_expand function. (no_warn_expand): Rewritten to muffle all warnings, and call no_ub_warn_expand. The constantp function continues to call no_warn_expand. (eval_init): Retarget expand intrinsic to no_ub_warn_expand, to preserve its documented behavior. The only function which currently uses no_warn_expand is constantp.
* New function: expand-hook-combine.Kaz Kylheku2025-04-013-0/+128
| | | | | | | | | | | | | This function provides a functional combinator that takes the responsibility of combining expand hooks. * eval.c (expand_hook_combine_fun, expand_hook_combine): New static functions. (eval_init): Register expand-hook-combine intrinsic. * tests/011/exphook.tl: New file. * txr.1: Documented.
* match: new pattern matching macro, match-tuple-case.Kaz Kylheku2025-04-014-2/+98
| | | | | | | | | | | | | * autolod.c (match_set_entries): Autoload match module on match-tuple-case. * match.tl (match-tuple-case): New macro. * tests/011/patmatch.tl: New tests. The macro is trivial; if lambda-match works, the macro works. * txr.1: Documented.
* New feature: *expand-hook*.Kaz Kylheku2025-03-313-1/+150
| | | | | | | | | | | | | | * eval.c (expand_hook_s): New symbol variable. (do_expand): Check for expand hook in several places and call it: symbol macros, macros, functions, and forms that not confirmed function calls. (eval_init): Initialize new symbol, and register the *expand-hook* special variable. * eval.h (expand_hook_s): Declared. (expand_hook): New macro. * txr.1: Documented.
* New function keep: generalized keepqual.Kaz Kylheku2025-03-285-0/+18
| | | | | | | | | | | * eval.c (eval_init): Register keep intrinsic. * lib.[ch] (keep): New function. * stdlib/compiler.tl (compiler comp-fun-form): Transform two argument keep to keepqual. * txr.1: Documented.
* compiler: reduce some equal-based sequence functions.Kaz Kylheku2025-03-281-0/+5
| | | | | | | | | | | | | | | * stdlib/compiler.tl (compiler comp-fun-form): Recognize two-argument forms of remove, count, pos, member and subst. When these don't specify test, key or map functions, they are equivalent to remqual, countqual, posqual, memqual and subqual. These functions are a bit faster because they have no arguments to default and some of their C implementations call the equal function either directly or via a pointer, rather than via going via funcall. The exceptions are posqual and subqual which actually call pos; but even for these it is still slightly advantageous to convert to to the fixed arity function, because funcall2 doesn't have to default the optional arguments with colon_k.
* New function: remove.Kaz Kylheku2025-03-275-5/+105
| | | | | | | | | | | | | | | We need a remove function that doesn't have an equality suffix, analogous to member, pos, count. * eval.c (eval_init): Register remove intrinsic. * lib.[ch] (remov): New function. Named this way to avoid clashing with the ISO C remove function in <stdlib.h>. * tests/012/seq.tl: New tests. * txr.1: Documented.
* case synonyms for more readable case macros.Kaz Kylheku2025-03-242-9/+41
| | | | | | | | | | | | | | | | There is a general trend in TXR Lisp to use a nice name for the variant of a function which uses equal equality. With the case macros, we have to use casequal, ecasequal, ecasequal or ecasequal*. Let's introduce synonyms for these: case, case*, ecase and ecase*. * eval.c (case_s, case_star_s): New symbol variables. (me_case): Check for case_star_s also to determine whether we have a "star" macro. (eval_init): Initialize symbol variables, and register case, case*, ecase and ecase* intrinsics. * txr.1: Documented.
* rand: rearrange code to test fixnump first.Kaz Kylheku2025-03-231-46/+46
| | | | | | * rand.c (random): The fixnump is going to come up more often than bignum, and is also fast to test for since fixnums are identified by just the tag in value word.
* rand: use PRNG bits more economically for small moduli.Kaz Kylheku2025-03-214-61/+225
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The rand function always calls rand32 at least one for each call, even for small moduli that need only a few pseudo-random bits. For instance when the modulus is 2, the function requires only one pseudo-random bit from the PRNG, yet it takes 32 and throws away 31. With this commit, that changes. For moduli of 65536 or smaller, the bits are used more efficiently; and for a modulus of 2, the function can satisfy 32 calls using the bits of a single rand32_t word: one stepping of the WELL512a PRNG. * rand.c (struct rand_state): New member, shift. Holds the shift register for rand/random to take bits from, replenished from rand32() when it runs out. The shift register detects when it runs out of bits in a clever way, without any additional variable. The register is regarded as being 33 bits wide, with a top bit that is always 1. When the register is empty, a 32 bit word is taken from the PRNG. The required random bits are taken from the word, and it is then shifted to the right. (We take only power-of-two amounts out of the shift register: 1, 2, 4, 8 or 16 bits). Even the smallest shift produces enough room that the 33rd bit can be added to the word, into its shifted position. After that, the shift register is considered to have enough bits for a given modulus if its value is less than equal to the mask. I.e if we were to take bits from it, we would be including the unconditional signaling bit. At that point we clobber the shift register with a new set of 32 bits from the PRNG, take the random bits we need, shift it to the right and add the signaling bit. (opt_noshift): New static variable; indicates whether we are in compatibility mode, requiring the shift register optimization to be defeated. (make_random_state): Initialize shift register to 0 in several places. (random): Implement various small modulus cases. There are specific cases for moduli that are exactly 65536, 256, 16, 4, 3 and 2. The in-between cases are handled by shifting the bits in the same amounts as the next higher power of two from this list of sizes: 16, 8, or 4 bits. For these cases, we calculate the smallest Mersenne modulus which covers the bits of the actual moduls and use that for rejecting potential values, just as we do in the general large modulus case. For instance if the modulus is 60 (range 0 to 59), that lands into the 8 bit shift range: we pull 8 bits at a time from the shift register. But the modulus 60 is covered by the six bit mask 63. We mask each 8 bit value with 63, and if it is in the required range 0 to 59, we accept it, otherwise draw another 8 bits. (rand_compat_fixup): Initialize opt_noshift to 1 if the requested compat version is 299 or less. * tests/012/sort.tl: Fix one test case involving shuffled data. The shufle function uses rand with small moduli, so its behavior changes for the same PRNG sequence. * tests/013/maze.expected: Likewise, the generated pseudo-random maze in the maze test case is different now; we must update to the new expected output. * txr.1: Document that a value of 299 or less of the compatibility -C option has an effect on rand.
* rand: eliminate small static function called only once.Kaz Kylheku2025-03-211-8/+2
| | | | | | * rand.c (make_state): Static function removed. (make_random_state): make_state logic integrated into this one and only caller.
* New macro: letrec.Kaz Kylheku2025-03-213-1/+211
| | | | | | | | | * eval.c (me_letrec): New function. (eval_init): Register letrec intrinsic macro. * tests/012/let.tl: New file. * txr.1: Documented, and also referenced from mlet.
* doc: document that let allows (var).Kaz Kylheku2025-03-211-4/+12
| | | | | | | | | | TXR Lisp's let and let* support var, (var) and (var init-form), exactly like Common Lisp, and this has been the case for a very long time. The (var) variant is now documented. * txr.1: Show the init-form in square brackets, making it clear that it's optional. Mention the (sym) variant in the documentation below.
generated by cgit v1.2.3 (git 2.25.1) at 2025-07-20 07:53:06 +0000