Skip to content

security/fail2ban: update to 1.1.0#175

Open
Shruthii7 wants to merge 1 commit into
NetBSD:trunkfrom
Shruthii7:fail2ban-1.1.0-update
Open

security/fail2ban: update to 1.1.0#175
Shruthii7 wants to merge 1 commit into
NetBSD:trunkfrom
Shruthii7:fail2ban-1.1.0-update

Conversation

@Shruthii7
Copy link
Copy Markdown

The upstream 1.1.0 release adds compatibility with Python 3.12 and 3.13, which resolves the missing asynchat module reported in issue #174.

Upstream 1.1.0 adds compatibility with Python 3.12/3.13,
which resolves the missing asynchat module issue.

Discussed in pkgsrc issue NetBSD#174.
@iamleot
Copy link
Copy Markdown
Member

iamleot commented Mar 16, 2026

@Shruthii7 can you please commit it via pkgsrc-wip?

Please see: https://pkgsrc.org/wip/users .

Thanks!

@Shruthii7
Copy link
Copy Markdown
Author

@iamleot Hi.
The update has been pushed to pkgsrc-wip.
I would appreciate any feedback or review.

Thanks.

@iamleot
Copy link
Copy Markdown
Member

iamleot commented Mar 19, 2026

@Shruthii7 Thank you!

netbsd-srcmastr pushed a commit that referenced this pull request Mar 26, 2026
2.51 2026-03-20 (by Todd Rinaldo)

  Fixes:
    - PR #184 GH #182 Fix compile warnings from Fedora gcc: unhandled enum
      value in switch and uninitialized variables in parse_stream
    - PR #186 GH #183 Add explicit case labels for XML_CTYPE_EMPTY and
      XML_CTYPE_ANY in generate_model switch to fix -Wswitch on BSD/clang
    - PR #175 GH #174 Add XML_DTD/XML_GE compile-time guards to AllocTracker
      and BillionLaughs XS bindings to fix warnings when expat lacks these flags
    - PR #187 GH #185 Require proper declaration in AllocTracker feature
      detection to prevent implicit function declaration on BSD

  Improvements:
    - PR #189 GH #188 Add tests for XS functions modified in PR #184 covering
      XML_ErrorString, XML_GetBase/XML_SetBase, and generate_model

  Maintenance:
    - PR #181 GH #180 Add Fedora 43 container job to CI test matrix
    - PR #178 Add BSD testing (FreeBSD, OpenBSD, NetBSD) to CI
    - PR #179 Add no-lwp CI job to validate test suite without LWP::UserAgent
    - Update MANIFEST

2.49 2026-03-19 (by Todd Rinaldo)

  Improvements:
    - PR #171 Expose expat library version at runtime via expat_version()
      and expat_version_info() class methods on XML::Parser::Expat
    - PR #169 GH #168 Expose Expat 2.7.2 AllocTracker security APIs
      (AllocTrackerMaximumAmplification, AllocTrackerActivationThreshold)

  Maintenance:
    - PR #170 Clean up build configuration: remove dead CAPI code from
      Makefile.PL and Expat/Makefile.PL, add test dependencies to cpanfile
khorben pushed a commit to NetBSDfr/NetBSD-pkgsrc that referenced this pull request Apr 14, 2026
1.6.1 (2026-03-05)

This is a gem housekeeping release. No user-facing changes.

Housekeeping

* Add Ruby 4 to CI (NetBSD#174) @mattbrictson
* Update minitest requirement from ~> 5.10 to ~> 6.0 (NetBSD#173) @dependabot[bot]
* Update mocha requirement from ~> 2.1 to ~> 3.0 (NetBSD#172) @dependabot[bot]
* Update rubocop gems (NetBSD#171, NetBSD#175, NetBSD#176) @dependabot
khorben pushed a commit to NetBSDfr/NetBSD-pkgsrc that referenced this pull request Apr 14, 2026
Version 6.1.2
=============

Code Fixes
----------

* Fix for low-severity CVE-2026-23868 affecting gifponge, giftool, and gifbuild,
  but not the core library - library clients need not be alarned.

Version 6.1.1
=============

This release bumps the major version, but only one entry point -
EGifSpew() - has changed signature and behavior (in order to be able
to pass out a detailed error code). The internal error
codes in the E_GIF_ERR series have changed value so none of them
collides with GIF_ERROR.

This code has been systematically audited and hardened wuth
ChatGPT-5.2. The only library fixes reported by users or found by
robot were for some memory leaks that could only triggered by severely
malformed GIFs. Other bugs are edge-case failures in the CLI tools.

The gif2rbg CLI tool has been moved to the "obsolete" bin, because its
only deployment case in 2026 is as a piñata at fuzzer parties.

Warning: the CLI tools in the obsolete category will soon be removed
from the distribution entirely. The maintainer is tired of fielding
junk bugs filed against them by would-be coup-counters who found yet
another edge case, and the rest of the world doesn't need noisy CVEs
that aren't actually DoS or security issues for giflib clients.

Code Fixes
----------

* Fix for CVE-2021-40633.

* Fix SF bug NetBSD#165 EGifSpew leaks GifFileOut->SColorMap

* Fix SF bug NetBSD#171 ImageMagick required to build giflib on non-Darwin Platforms

* Fix SF bug NetBSD#172 Incorrect object files in shared libutil on darwin

* Fix SF bug NetBSD#173 installation of manual pages and html documentation

* Fix SF bug NetBSD#175 Memory leaks in gifecho.c's main() and in gifalloc.c's GifMakeMapObject

* Fix SF bug NetBSD#177 wrong pointer used in giftool getbool

* Fix SF bug NetBSD#179 Path Traversal vulnerability

* Fix SF bug #180: -Wformat-truncation likely pointing out an actual bug

* Fix SF bug #182 out‐of‐bounds writes in Icon2Gif

* Fix SF bug #184 uninitialized buffer in DumpScreen2RGB

* Fix SF bug #185 integer overflow in gifbg.c

* Fix SF bug #186 integer overflow in Icon2Gif

* Fix SF bug #187: CVE-2025-31344

* Fix SF bug NetBSD#170 Tests failing on Ubuntu Noble, giftext buffer overflow

* Fix SF bug NetBSD#165 EGifSpew leaks GifFileOut->SColorMap

* Fix SF bug NetBSD#162 detected memory leaks in GifMakeSavedImage giflib/gifalloc.c

* Fix SF bug NetBSD#161 detected memory leaks in EGifOpenFileHandle giflib/egif_lib.c

* Fix SF bug NetBSD#142 ABI break public symbol GifQuantizeBuffer

Other bugs that duplicate these have breen addressesed by these fixes

* SF bug NetBSD#156 EGifSpew leaks SavedImages (and more); won't fix, caller
  might want to write a GIF, modify the in-memory data, then write
  again.

Tests
-----

Test suite now emits TAP (Test Anything Protocol).
msk pushed a commit to msk/pkgsrc that referenced this pull request May 11, 2026
[1.13.2] - 2025-01-06
Fixed

    Try another clipboard backend to try to fix cross platform issues. (Mag Mell, #226)
    rbw unlocked no longer starts the agent if it isn't running. (#223)
    The cardholder_name field is now correctly populated for card entries. (#204)
    Fix ip address url matching when using the domain match type. (#211)
    Make the behavior of matching urls with no paths when using the exact match type more consistent. (#211)

[1.13.1] - 2024-12-27
Fixed

    Moved clipboard support to a (default-enabled) feature, since not all platforms support it (disabling this feature should allow Android builds to work again).

[1.13.0] - 2024-12-26
Fixed

    Fix preventing the password type selectors in rbw generate from being used together. (antecrescent, #198)
    Fix --clipboard on Wayland (Maksim Karelov, #192)
    Fix parsing vaults with entries that have non-null field types (Tin Lai, #212)
    Fix lock timeout being reset when checking version (aeber, #216)
    Update API request headers to pass new stricter validation on the official bitwarden.com server (Davide Laezza, #219)
    Make it possible to start the rbw agent process from a graphical session and then access it over SSH (Wim de With, #221)

[1.12.1] - 2024-07-28
Fixed

    Fix decrypting folder names of entries with individual item encryption keys.

[1.12.0] - 2024-07-28

NOTE: If you were affected by issue NetBSD#163 (getting messages like failed to decrypt encrypted secret: invalid mac when doing any operations on your vault), you will need to rbw sync after upgrading in order to update your local vault with the necessary new data.
Fixed

    Support decrypting entries encrypted with invididual item encryption keys, which are now generated by default from the official Bitwarden clients. (NetBSD#163)
    Correctly handle lowercased and padded base32 TOTP secrets. (owl, #189)
    Make locking agent memory to RAM optional, since it appears to not always be available. (NetBSD#143)

[1.11.1] - 2024-06-26
Fixed

    Updated the prelogin API endpoint to use the identity API instead of the base API, to correspond with upcoming changes to the official Bitwarden server (see bitwarden/server#4206)

[1.11.0] - 2024-06-20
Added

    Support SSO login. (dezeroku, NetBSD#174)
    Added rbw search, which finds and displays the name of entries matching a given search term.
    Added --ignorecase as an option to several subcommands. (Maximilian Götsch, NetBSD#164)
    The JSON output given by --raw now also includes the field type.

Fixed

    Fixed the client id used when logging in, which was causing problems with the official Bitwarden server. (Merlin Marek, #186)
    Reworked rbw-pinentry-keyring to support passwords with spaces and 2fa codes. (Henk van Maanen, NetBSD#178)
    Try less hard to parse input as a url (so that using rbw get on an entry name containing a : works as expected).

[1.10.2] - 2024-05-20
Fixed

    Fix logging into the official Bitwarden server due to changes on their end (Gabriel Górski, NetBSD#175)

[1.10.1] - 2024-05-08
Added

    rbw code supports TOTP codes which use a SHA256 or SHA512 hash (Jonas, NetBSD#172)

Fixed

    Fix rbw code searching by UUID (Robert Günzler, NetBSD#169)
msk pushed a commit to msk/pkgsrc that referenced this pull request May 11, 2026
Snowball 3.0.1 (2025-05-09)
===========================

Python
------

* The __init__.py in 3.0.0 was incorrectly generated due to a missing
  build dependency and the list of algorithms was empty.  First reported by
  laymonage.  Thanks to Dmitry Shachnev, Henry Schreiner and Adam Turner for
  diagnosing and fixing.  (#229, #230, #231)

* Add trove classifiers for Armenian and Yiddish which have now been registered
  with PyPI.  Thanks to Henry Schreiner and Dmitry Shachnev.  (#228)

* Update documented details of Python 2 support in old versions.

Snowball 3.0.0 (2025-05-08)
===========================

Ada
---

* Bug fixes:

  + Fix invalid Ada code generated for Snowball `loop` (it was partly Pascal!)
    None of the stemmers shipped in previous releases triggered this bug, but
    the Turkish stemmer now does.

  + The Ada runtime was not tracking the current length of the string
    but instead used the current limit value or some other substitute, which
    manifested as various incorrect behaviours for code inside of `setlimit`.

  + `size` was incorrectly returning the difference between the limit and the
    backwards limit.

  + `lenof` or `sizeof` on a string variable generated Ada code that didn't
    even compile.

  + Fix incorrect preconditions on some methods in the runtime.

  + Fix bug in runtime code used by `attach`, `insert`, `<-` and string
    variable assignment when a (sub)string was replaced with a larger string.
    This bug was triggered by code in the Kraaij-Pohlmann Dutch stemmer
    implementation (which was previously not enabled by default but is now the
    standard Dutch stemmer).

  + Fix invalid code generated for `insert`, `<-` and string variable
    assignment.  This bug was triggered by code in the Kraaij-Pohlmann
    Dutch stemmer implementation (which was previously not enabled by default
    but is now the standard Dutch stemmer).

  + Generate valid code for programs which don't use `among`.  This didn't
    affect code generation for any algorithms we currently ship.

  + If the end of a routine was unreachable code the Snowball compiler
    would think the start of the next routine was also unreachable and would
    not generate it.  This didn't affect code generation for any algorithms we
    currently ship.

* Code quality:

  + Only declare variables A and C when each is needed.

  + Fix indentation of generated declarations.

  + Drop extra blank line before `Result := True`.

C/C++
-----

* Bug fixes:

  + Fix potential NULL dereference in runtime code if we failed to allocate
    memory for the p or S member for a Snowball program which uses one or more
    string variables.  Problem was introduced in Snowball 2.0.0.  Fixes #206,
    reported by Maxim Korotkov.

  + Fix invalid C code generated when a failure is handled in a context with
    the opposite direction to where it happened, for example:

        externals (stem)
        define stem as ( try backwards 'x' )

    This was fixed by changing the C generator to work like all the other
    generators and pre-generate the code to handle failure.

  + Eliminate assumptions that NULL has all-zero bit pattern.  We don't know
    of any current platforms where this assumption fails, but the C standard
    doesn't require an all-zero bit pattern for NULL.  Fixes #207.

* Optimisations:

  + Store index delta for among substring_i field.  This makes trying
    substrings after a failed match slightly faster because we can just add
    the offset to the pointer we already have to the current element.

* Code quality:

  + Improve formatting of generated code.

C#
--

* Bug fixes:

  + Add missing runtime support for testing for a string var at the current
    position when working forwards.  This situation isn't exercised by any of
    the stemming algorithms we currently ship.

  + Adjust generated code to work around a code flow analysis bug in the `mcs`
    C# compiler.

* Code quality:

  + Prune unused `using System.Text;`.

  + Generate C# with UTF-8 source encoding.  This makes the generated code
    easier to follow, which helps during development.  It's also a bit smaller.
    For now codepoints U+0590 and above are still emitted as escape sequences
    to avoid confusing source code rendering when LTR scripts are involved.

Go
--

* Optimisations:

  + Drop some unneeded Go code generated for string `$`.  None of the shipped
    stemmers use string `$`, though the Schinke Latin stemmer algorithm on the
    website does.

* Code quality:

  + Dispatch among result with `switch` instead of an `if` ... `else if` chain
    (which looks like we did because the Go generator evolved from the Python
    generator and Python didn't used to have a switch-like construct.  This
    doesn't make a measurable speed difference so it seems the Go compiler is
    optimising both to equivalent code, but using a switch here seems clearer,
    a better match for the intent, and is a bit simpler to generate.

  + Generate Go with UTF-8 source encoding.  This makes the generated code
    easier to follow, which helps during development.  It's also a bit smaller.
    For now codepoints U+0590 and above are still emitted as escape sequences
    to avoid confusing source code rendering when LTR scripts are involved.

Java
----

* The Java code generated by Snowball requires now requires Java >= 7.  Java 7
  was released in 2011, and Java 6's EOL was 2013 so we don't expect this
  to be a problematic requirement.  See #195.

* Optimisations:

  + We now store the current string in a `char[]` rather than using a
    `StringBuilder` to reduce overheads.  The `getCurrent()` method continues
    to return a Java `String`, but the `char[]` can be accessed using the new
    `getCurrentBuffer()` and `getCurrentBufferLength()` methods.  Patch from
    Robert Muir (#195).

  + Use a more efficient mechanism for calling `among` functions.  Patch from
    Robert Muir (#195).

* Code quality:

  + Consistently put `[]` right after element type for array types, which seems
    the most used style.

  + Fix javac warnings in SnowballProgram.java.

  + Improve formatting of generated code.

Javascript
----------

* Bug fixes:

  + Use base class specified by `-p` in string `$` rather than hard-coding
    `BaseStemmer` (which is the default if you don't specify `-p`).  None of
    the shipped stemmers use string `$`, though the Schinke Latin stemmer
    algorithm on the website does.

* Code quality:

  + Modernise the generated code a bit.  Loosely based on changes proposed in
    NetBSD#123 by Emily Marigold Klassen.

* Other changes:

  + The Javascript runner is now specified by make variable `JSRUN` instead
    of `NODE` (since node is just one JS implementation).  The default value
    is now `node` instead of `nodejs` (older Debian and Ubuntu packages used
    `/usr/bin/nodejs` because `/usr/bin/node` was already in use by a
    completely different package, but that has since changed).

Pascal
------

* Bug fixes:

  + Add missing semicolons to code generated in some cases for a function which
    always succeeds or always fails.  The new dutch.sbl was triggering this
    bug.

  + If the end of a routine was unreachable code the Snowball compiler
    would think the start of the next routine was also unreachable and would
    not generate it.  This didn't affect code generation for any algorithms we
    currently ship.

* Code quality:

  + Eliminate commented out code generated for string `$`.  None of the shipped
    stemmers use string `$`, though the Schinke Latin stemmer algorithm on the
    website does.

* Other changes:

  + Enable warnings, etc from fpc.

  + Select GNU-style diagnostic format.

Python
------

* Optimisations:

  + Use Python set for grouping checks.  This speeds up running the Python
    testsuite by about 4%.

  + Routines used in `among` are now referenced by name directly in the
    generated code, rather than using a string containing the name.  This
    avoids a `getattr()` call each time an among wants to call a routine.  This
    doesn't seem to make a measurable speed difference, but it's cleaner and
    avoids problems with name mangling.  Suggested by David Corbett in #217.

  + Simplify code generated for `loop`.  If the iteration count is constant and
    at most 4 then iterate over a tuple which microbenchmarking shows is
    faster.  The only current uses of loop in the shipped stemmers are `loop 2`
    so benefit from this.  Otherwise we now use `range(AE)` instead of
    `range (AE, 0, -1)` (the actual value of the loop variable is never
    used so only the number of iterations matter).

* Bug fixes:

  + Correctly handle stemmer names with an underscore.

* Code quality:

  + Generate Python with UTF-8 source encoding.  This makes the generated code
    easier to follow, which helps during development.  It's also a bit smaller.
    For now codepoints U+0590 and above are still emitted as escape sequences
    to avoid confusing source code rendering when LTR scripts are involved.

* Other changes:

  + Set python_requires to indicate to install tools that the generated code
    won't work with Python 3.0.x, 3.1.x and 3.2.x (due to use of `u"foo"`
    string literals).  Closes #192 and #191, opened by Andreas Maier.

  + Add classifiers to indicate support for Python 3.3 and for 3.8 to 3.13.
    Fixes NetBSD#158, reported by Dmitry Shachnev.

  + Stop marking the wheel as universal, which had started to give a warning
    message.  Patch from Dmitry Shachnev (#210).

  + Stop calling `setup.py` directly which is deprecated and now produces a
    warning - use the `build` module instead.  Patch from Dmitry Shachnev
    (#210).

Rust
----

* Optimisations:

  + Shortcut unnecessary calls to find_among, porting an optimization from the
    C generator.  In some stemming benchmarks this improves the performance
    of the rust english stemmer by about 27%.  Patch from jedav (#202).

* Code quality:

  + Suppress unused_parens warning, for example triggered by the code generated
    for `$x = x*x` (where `x` is an integer).

  + Dispatch `among` result with `match` instead of an `if` ... `else if` chain
    (which looks like we did because the Rust generator evolved from the Python
    generator and Python didn't used to have a switch-like construct.  This
    results in a 3% speed-up for an unoptimised Rust compile but doesn't seem
    to make a measurable difference when optimising so it seems the Rust
    compiler is optimising both to equivalent code.  However using a `match`
    here seems clearer, a better match for the intent, and is a bit simpler to
    generate.

  + Generate Rust with UTF-8 source encoding.  This makes the generated code
    easier to follow, which helps during development.  It's also a bit smaller.
    For now codepoints U+0590 and above are still emitted as escape sequences
    to avoid confusing source code rendering when LTR scripts are involved.

New stemming algorithms
-----------------------

* Add Esperanto stemmer from David Corbett (#185).

* Add Estonian algorithm from Linda Freienthal (NetBSD#108).

Behavioural changes to existing algorithms
------------------------------------------

* Dutch: Switch to Kraaij-Pohlmann as the default for Dutch.  In case you
  want Martin Porter's Dutch stemming algorithm for compatibility, this is now
  available as `dutch_porter`.  Fixes NetBSD#1, reported by gboer.

* Dutch (Kraaij-Pohlmann): Fix differences between the Snowball implementation
  and the original C implementation.

* Dutch (Kraaij-Pohlmann): Add a small number of exceptions to the Snowball
  implementation to avoid unwanted conflations.  This addresses all cases so
  far identified which Martin's Dutch stemmer handled better.  Fixes #208.

* Dutch (Porter): The "at least 3 characters" part of the R1 definition was
  actually implemented such that when working in UTF-8 it was "at least 3
  bytes".  We stripped accents normally found in Dutch except for `è` before
  setting R1, and no Dutch words starting `è` seem to stem differently
  depending on encoding, but proper nouns and other words of foreign origin may
  contain other accented characters and it seems better for the stemmer to
  handle such words the same way regardless of the encoding in use.

* English: Replace '-ogist' with '-og' to conflate "geologist" and "geology", etc.
  Suggested by Marc Schipperheijn on snowball-discuss.

* English: Add extra condition to undoubling.  We no longer undouble if the
  double consonant is preceded by exactly "a", "e" or "o" to avoid conflating
  "add"/"ad", "egg"/"eg", "off"/"of", etc.  Fixes #182, reported by Ed Page.

* English: Avoid conflating 'emerge' and 'emergency'.  Reported by Frederick Ross
  on snowball-discuss.

* English: Avoid conflating 'evening' and 'even'.  Reported by Ann B on
  snowball-discuss.

* English: Avoid conflating 'lateral' and 'later'.  Reported by Steve Tolkin on
  snowball-discuss.

* English: Avoid conflating 'organ', 'organic' and 'organize'.

* English: Avoid conflating 'past' and 'paste'.  Reported by Sonny on
  snowball-discuss.

* English: Avoid conflating 'universe', 'universal' and 'university'.  Reported
  by Clem Wang on snowball-discuss.

* English: Handle -eed and -ing exceptions in their respective rules.
  This avoids the overhead of checking for them for the majority of
  words which don't end -eed or -ing.  It also allows us to easily handle
  vying->vie and hying->hie at basically no extra cost.  Reduces the time to
  stem all words in our English word list by nearly 2%.

* French: Remove elisions as first step.  See #187.  Originally reported by
  Paul Rudin and kelson42.

* French: Remove -aise and -aises so for example, "française" and "françaises"
  are now conflated with "français".  Fixes #209.  Originally reported by
  ririsoft and Fred Fung.

* French: Avoid incorrect conflation of `mauvais` (bad) with `mauve` (mauve,
  mallow or seagull); avoid conflating `mal` with `malais`, `pal` with
  `palais`, etc.

* French: Avoid conflating `ni` (neither/nor) with `niais`
  (inexperienced/silly) and `nie`/`nié`/`nier`/`nierais`/`nierons` (to deny).

* French: -oux -> -ou.  Fixes NetBSD#91, reported by merwok.

* German: Replace with the "german2" variant.  This normalises umlauts ("ä" to
  "ae", "ö" to "oe", "ü" to "ue") which is presumably much less common in
  newly created text than it once was as modern computer systems generally
  don't have the limitations which motivated this, but there will still be
  large amounts of legacy text which it seems helpful for the stemmer to
  handle without having to know to select a variant.

  On our sample German vocabulary which contains 35033 words, 77 words give
  different stems.  A significant proportion of these are foreign words, and
  some are proper nouns.  Some cases definitely seem improved, and quite a few
  are just different but effectively just change the stem for a word or group
  of words to a stem that isn't otherwise generated.  There don't seem any
  changes that are clearly worse, though there are some changes that have both
  good and bad aspects to them.

  Fixes NetBSD#92, reported by jrabensc.

* German: Don't remove -em if preceded by -syst to avoid overstemming words
  ending -system.  This change means we now conflate e.g. "system" and
  "systemen".  Partly addresses NetBSD#161, reported by Olga Gusenikova.

* German: Remove -erin and -erinnen suffixes which conflates singular and
  plural female versions of nouns with the male versions.  Fixes NetBSD#85 and
  partly addresses NetBSD#161, reported by Olga Gusenikova.

* German: Replace -ln and -lns with -l.  This improves 82 cases in the current
  sample data without making anything worse.  Tests on a larger word list look
  good too.  Partly addresses NetBSD#161, reported by Olga Gusenikova.

* German: Remove -et suffix when we safely can.  Fixes #200, reported by Robert
  Frunzke.

* Greek: Fix "faulty slice operation" for input `ισαισα`.  The fix changes
  `ισα` to stem to `ισ` instead of the empty string, which seems better (and to
  be what the second paper actually says to do if read carefully).  Fixes #204,
  reported by subnix.

* Italian: Address overstemming of "divano" (sofa) which previously stemmed to
  "div", which is the stem for 'diva' (diva).  Now it is stemmed to 'divan',
  which is what its plural form 'divani' already stemmed to.  Fixes NetBSD#49,
  reported by francesco.

* Norwegian: Improve stemming of words ending -ers.  Fixes NetBSD#175, reported by
  Karianne Berg.

* Norwegian: Include more accented vowels - treating "ê", "ò", "ó" and "ô"
  as vowels improves the stemming of a fairly small number of words, but
  there's basically no cost to having extra vowels in the grouping, and some
  of these words are commonly used.  Fixes #218, reported by András Jankovics.

* Romanian: Fix to work with Romanian text encoded using the correct Unicode
  characters.  Romanian uses a "comma below" diacritic on letters "s" and "t"
  ("ș" and "ț").  Before Unicode these weren't easily available so Romanian
  text was written using the visually similar "cedilla" diacritic on these
  letters instead ("ş" and "ţ").  Previously our stemmer only recognised the
  latter.  Now it maps the cedilla forms to "comma below" as a first step.
  Patch from Robert Muir.

* Spanish: Handle -acion like -ación and -ucion like -ución.  It's apparently
  common to miss off accents in Spanish, and there are examples in our test
  vocabulary that these change helps.  Proposed by Damian Janowski.

* Swedish: Replace suffix "öst" with "ös" when preceded by any of 'iklnprtuv'
  rather than just 'l'.  The new rule only requires the "öst" to be in R1
  whereas previously we required all of "löst" to be.  This second tweak
  doesn't seem to affect any words ending "löst" but it conflates a few extra
  cases when combined with the expanded list of preceding letters, and seems
  more logical linguistically (since "ös" is akin to "ous" in English).  Fixes
  NetBSD#152, reported by znakeeye.

* Swedish: Remove -et/-ets in cases where it helps.  Removing -et can't be done
  unconditionally because many words end in -et where this isn't a suffix.
  However it's a very common suffix so it seems worth crafting a more complex
  condition under which to remove.  Fixes NetBSD#47.

* Turkish: Remove proper noun suffixes.  For example, `Türkiye'dir` ("it is
  Turkey") is now conflated with `Türkiye` ("Turkey").  Fixes #188.

* Yiddish: Avoid generating empty stem for input "גע" (not a valid word, but
  it's better to avoid an empty stem for any non-empty input).

Optimisations to existing algorithms
------------------------------------

* General change: Use `gopast` everywhere to establish R1 and R2 as it is a
  little more efficient to do so.

* Basque: Use an empty action rather than replacing the suffix with itself
  which seems clearer and is a little more efficient.

* Dutch (Porter): Optimise prelude routine.

* English: Remove unnecessary exception for `skis` as the algorithm stems
  `skis` to `ski` by itself (`skies` and `sky` do still need a special case to
  avoid conflation with `ski` though).

* Hungarian: We no longer take digraphs into account when determining where R1
  starts.  This can only make a difference to the stemming if we removed a
  suffix that started with the last character of the digraph (or with "zs" in
  the case of "dzs"), and that doesn't happen for any of the suffixes we remove
  for any valid Hungarian words.  This simplification speeds up stemming by
  ~2% on the current sample vocabulary list.  See #216.  Thanks to András
  Jankovics for confirming no Hungarian words are affected by this change.

* Lithuanian: Remove redundant R1 check.

* Nepali: Eliminate redundant check_category_2 routine.

* Tamil: Optimise by using `among` instead of long `or` chains.  The generated
  C version now takes 43% less time to processes the test vocabulary.

* Tamil: Remove many cases which can't be triggered due to being handled by
  another case.

* Tamil: Clean up some uses of `test`.

* Tamil: Make `fix_va_start` simpler and faster.

* Tamil: Localise use of `found_a_match` flag.

* Tamil: Eliminate pointless flag changes.

* Turkish: Minor optimisations.

Code clarity improvements to existing algorithms
------------------------------------------------

* Stop noting dates changes were made in comments in the code - we now maintain
  a changelog in each algorithm's description page on the website (and the
  version control history provides a finer grained view).

* Always use `insert` instead of `<+` as the named command seems clearer.

* English: Add comments documenting motivating examples for all exceptional
  cases.

* Lithuanian: Change to recommended latin stringdef codes.  Using common codes
  makes it easier to work across algorithms, but they are more mnemonic so also
  seem clearer when just considering this one algorithm.

* Serbian: Change to recommended latin stringdef codes.  Using common codes
  makes it easier to work across algorithms, but they are more mnemonic so also
  seem clearer when just considering this one algorithm.

* Turkish: Use `{sc}` for s-cedilla and `{i}` for dotless-i to match other
  uses.

Compiler
--------

* Generic code generation improvements:

  + Show Snowball source leafname in "generated" comment at start of files.

  + Add generic reachability tracking machinery.  This facilitates various new
    optimisations, so far the following have been implemented:

    - Tail-calling
    - Simpler code for calling routines which always give the same signal
    - Simpler code when a routine ends in a integer test (this also allows
      eliminating an Ada-specific codegen optimisation which did something
      similar but only for routines which consisted *entirely* of a single
      integer test.
    - Dead code reporting and removal (only in simple cases currently)

    Currently this overlaps in functionality with the existing reachability
    tracking which is implemented on a per-language basis, and only for some
    languages.  This reachability tracking was originally added for Java
    where some unreachable code is invalid and result in a compile time error,
    but then seems to have been copied for some other newer languages which
    may or may not actually need it.  The approach it uses unfortunately
    relies on correctly updating the reachability flag anywhere in the
    generator code where reachability can change which has proved to be a
    source of bugs, some unfixed.  This new approach seems better and with some
    more work should allow us to eliminate the older code.  Fixes NetBSD#83.

  + Omit check for `among` failing in generated code when we can tell at
    compile time that it can't fail.

  + Optimise `goto`/`gopast` applied to a grouping or inverted grouping (which
    is by far the most common way to use `goto`/`gopast`) for all target
    languages (new for Go, Java, Javascript, Pascal and Rust).

  + We never need to restore the cursor after `not`.  If `not` turns signal `f`
    into `t` then it sets `c` back to its old position; otherwise, `not`
    signals `f` and `c` will get reset by whatever ultimately handles this `f`
    (or the program exits and the position of `c` no longer matters).  This
    slightly improves the generated code for the `english` and `porter`
    stemmers.

  + Don't generate code for undefined or unused routines.

  + Avoid generating variable names and then not actually using them.  This
    eliminates mysterious gaps in the numbering of variables in the generated
    code.

  + Eliminate `!`/`not` from integer test code by generating the inverse
    comparison operator instead for all languages, e.g. for Python we now
    generate

      if self.I_p1 >= self.I_x:

    instead of

      if not self.I_p1 < self.I_x:

    This isn't going to be faster in compiled languages with an optimiser but
    for scripting languages it may be faster, and even if not, it makes for a
    little less work when loading the script.

  + Canonicalise `hop 1` to `next` as the generated code for `next` can be
    slightly more efficient.  This will also apply to `hop` followed by a
    constant expression which Snowball can reduce to `1`.

  + Avoid trailing whitespace in generated files.

  + Fix problems with --comments option:

    - When generating C code we would segfault for code containing `atleast`,
      `hop` or integer tests.
    - Fix missing comments for some commands in some target languages.
    - Fix inconsistent formatting of comments in some target languages.
    - Comments in C are now always on their own line - previously some were
      after at the end of the line and some on their own line which made them
      harder to follow.
    - Emit comments before `among` and before routine/external definitions.

  + Simplify more cases of numeric expressions (e.g. `x * 1` to `x`).

* Improve --help output.

* Division by zero during constant folding now gives an error.

* For `hop` followed by an unexpected token (e.g. `hop hop`) we were
  already emitting a suitable error but would then segfault.

* Emit error for redefinition of a grouping.

* Improve errors for `define` of an undeclared name.  We already peek at the
  next token to decide whether to try to parse as a routine or grouping.
  Previously we parsed as a routine if it was `as`, and a grouping otherwise,
  but routine definitions are more common and a grouping can only start with
  a literal string or a name, so now we assume a routine definition with a
  missing `as` if the next token isn't valid for either.

* Suppress duplicate (or even triplicate) "unexpected" errors for the same
  token when the compiler tried to recover from the error by adjusting the
  parse stare and marking the token to be reparsed, but the same token then
  failed to parse in the new state.

* Fix NULL pointer dereference if an undefined grouping is used in the
  definition of another grouping.

* Fix mangled error for `set` or `unset` on a non-boolean:

  test.sbl:2: nameInvalid type 98 in name_of_type()

* Emit warning if `=>` is used.  The documentation of how it works doesn't
  match the implementation, and it seems it has only ever been used in the
  Schinke stemmer implementation (which assumes the implemented behaviour).
  We've updated the Schinke implementation to avoid it.  If you're using it
  in your own Snowball code please let us know.

* Improve errors for unterminated string literals.

* Fix NULL pointer dereference on invalid code such as `$x = $y`.

* If malloc fails while compiling the compiler will now report the failure
  and exit.  Previously the NULL return from malloc wasn't checked for so
  we'd typically segfault.

* `lenof` and `sizeof` applied to a string variable now mark the variable
  as used, which avoids a bogus error followed by a confusing additional
  message if this is the only use of that variable:

  lenofsizeofbug.sbl:3: warning: string 's' is set but never used
  Unhandled type of dead assignment via sizeof

  This is situation is unlikely to occur in real world code.

* The reported line number for "string not terminated" error was one too high
  in the case where we were in a stringdef (but correct if we weren't).

* Eliminate special handling for among starter.  We now convert the starter
  to be a command before the among, adding an explict substring if there
  isn't one.

* We now warn if the body of a `repeat` or `atleast` loop always signals
  `t` (meaning it will loop forever which is very undesirable for a stemming
  algorithm) or always signals `f` (meaning it will never loop, which seems
  unlikely to be what was intended).

* Release memory in compiler before exit.  The OS will free all allocated
  memory when a process exits, so this memory isn't actually leaked, but it can
  be annoying with when using snowball as part of a larger build process with
  some leak-finding tools.  Patch from jsteemann in NetBSD#166.

* Store textual data more efficiently in memory during Snowball compilation.
  Previously almost all textual data was stored as 16 bit values, but most
  such data only uses 8 bit character values.  Doubling the memory usage
  isn't really an issue as Snowball programs are tiny, but this also
  complicated code handling such data.  Now only literal strings use the
  16 bit values.

* Fix clang -Wunused-but-set-variable warning in compiler code.

* Fix a few -Wshadow warnings in compiler and enable this warning by default.

* Tighten parsing of `writef()` format strings.  We now error out on
  unrecognised escape codes or if a numbered escape is used with too high a
  number or a non-digit.  This change reveals that the Go and Rust generators
  were using invalid escape ~A - the old writef() code was substituting this
  with just A which is what is wanted so this case was harmless but being
  lenient here could hide bugs, especially when copying code between
  generators as they don't all support the same set of format codes.

Build system
------------

* Turn on Java warnings and make them errors.

* Compile C code with -g by default.  This makes debugging easier, and
  matches the default for at least some other build systems (e.g. autotools).

* Fix "make clean" to remove all built Ada files.

* Clean `stemtest` too.  Patch from Stefano Rivera.

* Add missing `COMMON_FILES` dependency to dist targets.

* GNUmakefile: Tidy up and make more consistent

* GNUmakefile: Make use of $* to improve speed and readability.

* Use $(patsubst ...) instead of sed in .java.class rule which gives cleaner
  make output and is a bit more efficient.

* Add `WERROR` make variable to provide a way to add `-Werror` to existing
  CFLAGS.

libstemmer
----------

Testsuite
---------

* Give a clear error if snowball-data isn't found.  Fixes #196, reported by
  Andrea Maccis.

* Handle not thinning testdata better.  If THIN_FACTOR is set to 1 we no longer
  run gzipped test data through awk.  We also now handle THIN_FACTOR being set
  empty as equivalent to 1 for convenience.

* csharp_stemwords: Correctly handle a stemmer name containing an underscore.

* csharp_stemwords: Make `-i` option optional and read from stdin if omitted,
  like the C version does.

* csharp_stemwords: Process the input line by line which is more helpful for
  interactive testing, and also a little faster.

* Fix Java TestApp to allow a single argument.  The documented command line
  syntax is that you only need to specify the language and there was already
  code to read from stdin if no input file was specified, but at least two
  command line options were required.

* Fix deprecation warning in TestApp.java.

* Optimise TestApp.java by creating fewer objects.  Patch from Robert Muir.

* stemwords.py: We no longer create an empty output file if we fail to open the
  input file.

* stemwords: Improve error message to say "Out of memory or internal error"
  rather than just "Out of memory".

Documentation
-------------

* Include "what is stemming" section in each README.

* Include section on threads in each README.  Based on patch for Python from
  dbcerigo.

* Document that input should be lowercase with composed accents.  See #186,
  reported by 1993fpale.

* Add README section on building, including notes on cross-compiling.  Fixes
  #205, reported by sin-ack.

* CONTRIBUTING.rst: Clarify which charsets to list

* CONTRIBUTING.rst: Add general advice section.  In particular, note to use
  spaces-only for indentation in most cases.  Thanks to Dmitry Shachnev for
  raising this point.

* CONTRIBUTING.rst: Note that UTF-8 is OK in comments.  Thanks to Dmitry
  Shachnev for asking.

* Fix some typos.  Patch from Josh Soref.

* Document that our CI now uses github actions.

* Update link to Greek stemmer PDF.  Patch from Michael Bissett (NetBSD#33).

Snowball 2.2.0 (2021-11-10)
===========================

New Code Generators
-------------------

* Add Ada generator from Stephane Carrez (NetBSD#135).

Javascript
----------

* Fix generated code to use integer division rather than floating point
  division.

  Noted by David Corbett.

Pascal
------

* Fix code generated for division.  Previously real division was used and the
  generated code would fail to compile with an "Incompatible types" error.

  Noted by David Corbett.

* Fix code generated for Snowball's `minint` and `maxint` constant.

Python
------

* Python 2 is no longer actively supported, as proposed on the mailing list:
  https://lists.tartarus.org/pipermail/snowball-discuss/2021-August/001721.html

* Fix code generated for division.  Previously the Python code we generated
  used integer division but rounded negative fractions towards negative
  infinity rather than zero under Python 2, and under Python 3 used floating
  point division.

  Noted by David Corbett.

Code quality Improvements
-------------------------

* C/C++: Generate INT_MIN and INT_MAX directly, including <limits.h> from
  the generated C file if necessary, and remove the MAXINT and MININT macros
  from runtime/header.h.

* C#: An `among` without functions is now generated as `static` and groupings
  are now generated as constant.  Patches from James Turner in NetBSD#146 and NetBSD#147.

Code generation improvements
----------------------------

* General:

  + Constant numeric subexpressions and constant numeric tests are now
    evaluated at Snowball compile time.

  + Simplify the following degnerate `loop` and `atleast` constructs where
    N is a compile-time constant:

    - loop N C where N <= 0 is a no-op.

    - loop N C where N == 1 is just C.

    - atleast N C where N <= 0 is just repeat C.

    If the value of N doesn't depend on the current target language, platform
    or Unicode settings then we also issue a warning.

Behavioural changes to existing algorithms
------------------------------------------

* german2: Fix handling of `qu` to match algorithm description.  Previously
  the implementation erroneously did `skip 2` after `qu`.  We suspect this was
  intended to skip the `qu` but that's already been done by the substring/among
  matching, so it actually skips an extra two characters.

  The implementation has always differed in this way, but there's no good
  reason to skip two extra characters here so overall it seems best to change
  the code to match the description.  This change only affects the stemming of
  a single word in the sample vocabulary - `quae` which seems to actually be
  Latin rather than German.

Optimisations to existing algorithms
------------------------------------

* arabic: Handle exception cases in the among they're exceptions to.

* greek: Remove unused slice setting, handle exception cases in the among
  they're exceptions to, and turn `substring ... among ...  or substring ...
  among ...` into a single `substring ... among ...` in cases where it is
  trivial to do so.

* hindi: Eliminate the need for variable `p`.

* irish: Minor optimisation in setting `pV` and `p1`.

* yiddish: Make use of `among` more.

Compiler
--------

* Fix handling of `len` and `lenof` being declared as names.

  For compatibility with programs written for older Snowball versions
  len and lenof stop being tokens if declared as names.  However this
  code didn't work correctly if the tokeniser's name buffer needed to
  be enlarged to hold the token name (i.e. 3 or 5 elements respectively).

* Report a clearer error if `=` is used instead of `==` in an integer test.

* Replace a single entry command list with its contents in the internal syntax
  tree.  This puts things in a more canonical form, which helps subsequent
  optimisations.

Build system
------------

* Support building on Microsoft Windows (using mingw+msys or a similar
  Unix-like environment).  Patch from Jannick in NetBSD#129.

* Split out INCLUDES from CPPFLAGS so that CPPFLAGS can now be overridden by
  the user if required.  Fixes NetBSD#148, reported by Dominique Leuenberger.

* Regenerate algorithms.mk only when needed rather than on every `make` run.

libstemmer
----------

* The libstemmer static library now has a `.a` extension, rather than `.o`.
  Patch from Michal Vasilek in NetBSD#150.

Testsuite
---------

* stemtest: Test that numbers and numeric codes aren't damaged by any of the
  algorithms.  Regression test for NetBSD#66.  Fixes NetBSD#81.

* ada: Fix ada tests to fail if output differs.  There was an extra `| head
  -300` compared to other languages, which meant that the exit code of `diff`
  was ignored.  It seems more helpful (and is more consistent) not to limit how
  many differences are shown so just drop this addition.

* go: Stop thinning testdata.  It looks like we only are because the test
  harness code was based on that for rust, which was based on that for
  javascript, which was only thinning because it was reading everything into
  memory and the larger vocabulary lists were resulting in out of memory
  issues.

* javascript: Speed up stemwords.js.  Process input line-by-line rather than
  reading the whole file into memory, splitting, iterating, and creating an
  array with all the output, joining and writing out a single huge string.
  This also means we can stop thinning the test data for javascript, which we
  were only doing because the huge arabic test data file was causing out of
  memory errors.  Also drop the -p option, which isn't useful here and
  complicates the code.

* rust: Turn on optimisation in the makefile rather than the CI config.  This
  makes the tests run in about 1/5 of the time and there's really no reason to
  be thinning the testdata for rust.

Documentation
-------------

* CONTRIBUTING.rst: Improve documentation for adding a new stemming algorithm.

* Improve wording of Python docs.
msk pushed a commit to msk/pkgsrc that referenced this pull request May 11, 2026
# upstream changes (since 2.9.2)

## keychain 2.9.5 (16 May 2025)

This is a bugfix release.

* Hardening checks were failing on Android and some MacOS environments. Make them
  more compatible and lower to warnings instead of aborting the script, until
  they have been tested in more environments.
  ([NetBSD#177](danielrobbins/keychain#177))

* Fixed issues with indentation of `note()`, `warn()`, `mesg()`.

* Convert `SSH_AUTH_SOCK in pidfile is invalid; ignoring it` into a debug message,
  as this is normal when rebooting your system so is not really useful to show
  typically. ([NetBSD#176](danielrobbins/keychain#176))

## keychain 2.9.4 (14 May 2025)

This is a minor bugfix release.

* Fix minor regression which allowed some warnings to display with `--quiet`.
([NetBSD#175](danielrobbins/keychain#175))

* "Cannot find separate public key" turned into a `note()` rather than `warn()`,
  along with several other non-critical notices. `note()` can be suppressed with
  `--quiet`, unlike `warn()`. ([NetBSD#157](danielrobbins/keychain#157))

* Minor improvement when wiping GnuPG keys with `--wipe` option so keychain output
  is more understandable when gpg-agent is not running.

## keychain 2.9.3 (14 May 2025)

This is a security and bug fix release. Many thanks to those who have reported
issues to GitHub, send in pull requests, and tested out fixes. 2.9.3 includes
the following updates:

* The `--quick` option logic had several bugs which have been resolved. Thanks
  to Filipe Fernandes (@ffernand) for reporting the issue and for assistance
  testing fixes. ([NetBSD#167](danielrobbins/keychain#167))

* Fix keychain `--query` exit code when no pidfile exists.
  ([NetBSD#171](danielrobbins/keychain#171))

* `--systemd` option should now be fixed.
  ([[NetBSD#168](danielrobbins/keychain#168)])

* Harden keychain so the use of the `--dir` and `--absolute` options cannot be
  used to instruct keychain to write pidfiles into insecure areas.
  ([NetBSD#174](danielrobbins/keychain#174))

  Prior to this release, it was possible to use these options in combination
  with bad (empty) default umask to write pidfiles into a public area on disk
  where they were writable by other users. In the worst case, this could allow
  arbitrary execution of the contents of the malicious pidfile by keychain.

  This hardening now makes it difficult for a user to configure their keychain
  in a way that would allow this to happen. Note that if you are not using the
  `--dir` or `--absolute` options, keychain will use the `$HOME/.keychain`
  directory by default, which is typically under the full control of the
  current user and thus not exploitable.

  The hardening changes include:

  * Setting a global restrictive `umask` in the script.
  * Remove pidfiles before redirecting data to them to ensure they are created
    with restrictive permissions from the `umask`.
  * Check the keychain pidfile directory to ensure it is owned by the current
    user, and only the current user can access it (mode 700). If not, abort
    with an informative error message.
  * Check any existing pidfiles prior to use to make sure they are owned by the
    current user, and only the current user can access them. If not, abort with
    an informative error message.
msk pushed a commit to msk/pkgsrc that referenced this pull request May 11, 2026
Mew 6.10 (2025-07-08)

Differences between Mew 6.10 and Mew 6.9

* XOAUTH2 support. Please see Sec 9.13 of the manual.
* Use C.UTF-8 if available to display non-ASCII names for GnuPG programs.
  NetBSD#179
* Enable IPv6 support of stunnel by default.  NetBSD#177
* Add GnuTLS support.  NetBSD#175
* Supporting stunnel >=5.15 only.  NetBSD#174
* Separately check if stunnel supports 'foreground', 'pid' and 'syslog'
  options.  NetBSD#171
* Use configured file names as relative paths from mew-home.  NetBSD#170
* And several bug fixes.
msk pushed a commit to msk/pkgsrc that referenced this pull request May 11, 2026
Prepare for this replacing mail/getmail.

Version 6.19.08
14.May.2025
    - fix NetBSD#105 by converting empty string to None, which has smtplib handling
    - fix #226 by handling all Exception instead of just OSError on SIGCHLD to parent
    - fix #239 by catching ssl.SSLError
    - Merge pull request #237 from meeuw/integration-tests
    - remove python 2 code: only supporting python 3
    - Merge pull request #238 from yshingai/v6.19.07-add-alias
    - support windows-874 cp874
    - Merge pull request #234 from meeuw/fix-incorrect-fsf-address
    - Merge pull request #233 from meeuw/fix-license
    - #220 allow uid_cache=true (latest UID from oldmail-file) in addition to uid_cache=<uidfile>

Version 6.19.07
26.January.2025
    - #225 Missing comma in getmailcore/message.py

Version 6.19.06
18.January.2025
    - #223 publish to pypi workflow
    - #219 #220 increase IMAP efficiency through uid_cache  (needs to be configured) storing last fetched UID
    - updated tests
    - update copyright to 2025

Version 6.19.05
02.October.2024
    - fix #211: --only-oldmail-file command line = only_oldmail_file config: regenerate oldmail file with mails currenlty on server
    - fix #212: oldmail-* reverts when --to-oldmail-on-each-mail is used
    - Pyp2 fixes (#210)
    - fix #209: using the suggested simpler fix as pathlib is not used so far
    - build and upload sdist
    - Fix invalid SPDX expression

Version 6.19.04
25.August.2024
    - Update getmailrc-examples (#204, #206, #207)
    - #205: pop3: Support use_xoauth2

Version 6.19.03
08 July 2024
    - #199 fix: fails with "TypeError: IMAP4_SSL.__init__()" after changing to Python 3.12 (when keyfile/certfile in config)

Version 6.19.02
01 July 2024
    - #196 fix: handle KeyError (getmail-gmail-xoauth-tokens)
    - #197 fix: detect if --port is set to force the use of the local http server (getmail-gmail-xoauth-tokens)

Version 6.19.01
7 June 2024
    - #195 fix: make mark_read option act same as -ds,

Version 6.19.00
31 May 2024
    - #191 -m/--mark-read and mark_read in [options] of rcfile
    - #190 -o/--only-account to choose rcfile by username (=email)
    - #189 IMAP retrievers support imap_id_extension
    - NetBSD#137 Add mailbox name to log line

Version 6.18.14
20 March 2024
    - fix make lint
    - fix #183: in case of TypeError keyfile and certfile are used later in wrap_socket (Python 3.12)
    - fix #182: make log to trace
    - Allow to customize 'redirect_uri' (pull request NetBSD#179 from jerome-pouiller/master)
    - fix NetBSD#176: Maildir Filename Generation Issue with Truncated Hostname (use full hostname)
    - remove "Labels" before deleting uid (pull request NetBSD#175 from ewft/fix_gmail_deletion)
    - Redact real email in test data (pull request NetBSD#170 from h3xx/remove-email)
msk pushed a commit to msk/pkgsrc that referenced this pull request May 11, 2026
Changelog:
Version 0.8.0 (2025-05-04):
	* A backwards-incompatible change, since the sizeof cpu_raw_data_t and
	  cpu_id_t are now different.
	* Fix a typo in string (#208)
	* Support for AMD Turin Dense
	* Simplify exec_cpuid() to use constraints (#209)
	* Use HW_NCPUONLINE in get_total_cpus() for OpenBSD/NetBSD (#210)
	* Implement cpu_clock_by_os() for OpenBSD (#211)
	* Support Intel Lunar Lake
	* Support Intel Twin Lake
	* Support Intel Alder Lake-PS
	* Support more Intel Raptor Lake ("Intel Processor", "Xeon E")
	* Support Intel Raptor Lake-U Refresh
	* Support Intel Raptor Lake-U/H Re-refresh
	* Support Intel Arrow Lake-U/H
	* Fix warning during build on OpenBSD about strcpy()
	* Refactored internal struct match_entry_t for x86 CPUs to fix truncation issues (#212)
	* Add technology_node field to struct cpu_id_t
	* Support more ARM CPUs
	* Support ARMv9.5-A CPUs (new value FEATURE_LEVEL_ARM_V9_5_A for cpu_feature_level_t)
	* Massive DB updates for almost all Intel and AMD CPUs along with more test data
	* Return ERR_BADFMT during raw deserialization if cpu_raw_data_t is empty
	* Support another type of header for raw deserialization
	* Support Intel Granite Rapids-SP


Version 0.7.1 (2024-11-30):
	* Fix cpuid kernel module build on ARM
	* Return ERR_NO_CPUID when cpuid kernel module cannot be used on AArch32 state
	* Refactor the build of the bindings (#203)
	* Fix build error on AArch64 when HWCAP_CPUID is not defined (#205)
	* Fix build on Windows ARM with MSVC (206)
	* Fix detection of Intel Meteor Lake
	* Support for Intel Arrow Lake
	* Support for AMD Turin


Version 0.7.0 (2024-08-26):
	* Fix handle leaks in rdmsr.c (#199)
	* Fix cpuid_get_hypervisor when NULL data is provided (#199)
	* Prevent intel_fn11 array overruns (#199)
	* Support for AMD Hawk Point
	* Support for more AMD Phoenix (8000 series)
	* Add cpu_clock_by_tsc() function to the library (NetBSD#124)
	* Check x86 CPUs MSR support (#185)
	* Add support for ARM CPUs (AArch32 + AArch64) (#200)
	* Add cpu_feature_level_t enumerated values for x86 CPUs (NetBSD#177)
	* Support up to 4 subleaf entries for CPUID leaf 0x80000026 (#189)
	* Support for Extended CPU topology subleaf in cpuid_identify_purpose_amd() (#189)
	* Support CPU purpose for AMD x86 CPUs (#189)
	* Add cpuid Linux and FreeBSD kernel modules for ARM CPUs
	* Improve errors handling in cpuid_get_all_raw_data() and cpuid_get_raw_data_core() (#202)
	* Support get_total_cpus() on DragonFly BSD
	* Improve set_cpu_affinity() on NetBSD
	* Fix build on OpenBSD
	* Improve behavior when CPU affinity cannot be set
	* Fix a regression in cpuid_tool about arguments doing nothing (like --rdmsr or --cpuid)
	* Fix a segmentation fault when using --quiet in cpuid_tool
	* Improve error handling in cpu_identify_all()
	* Add Python bindings (#197)
	* Support for AMD Granite Ridge
	* Support for AMD Strix Point
	* Detect x2APIC and AVX512 features for AMD x86 CPUs


Version 0.6.5 (2024-04-28):
	* Support for Intel Bay Trail-M
	* Support for Intel Bay Trail-T
	* Support for Intel Bay Trail-D
	* Support for AMD Storm Peak
	* Support for Intel Raport Lake Refresh
	* Support heterogeneous RAW dumps in cpu_identify_all()
	* Support for Intel Meteor Lake, including detection of LP E-Cores
	* Support for Intel Emerald Rapids-SP
	* Support for more AMD Van Gogh


Version 0.6.4 (2023-10-08):
	* Support for AMD Genoa
	* Support for Intel Sapphire Rapids-WS
	* Support for Intel Skylake (server)
	* Support for Intel Cascade Lake
	* Support for Intel Ice-Lake (server)
	* Support for Intel Sapphire Rapids-SP
	* Improve support for Intel Arrandale (Pentium and Celeron)
	* Fix support for Intel Sandy Bridge-E (Core i7)
	* Support for AMD Phoenix
	* Support for Intel Raptor Lake-H/HX
	* Improve support for Intel Alder-Lake and Raptor-Lake P/U
	* Fix floating point exception in cpu_identify_all() for unsupported CPU vendor
	* Initial support for Centaur CPUs (VIA and Zhaoxin)
	* Support for Intel Apollo Lake
	* Fix build for NetBSD and DragonFly BSD (#190)
	* Support for AMD Dragon Range
	* Support for AMD Ryzen Z1


Version 0.6.3 (2023-04-02):
	* Support for Intel Pentium and Celeron for Alder Lake-S
	* Support for Intel Alder Lake-HX
	* Support for Intel Alder Lake-X
	* Fix detection of Intel Alder Lake-P
	* Fix infinite loop in set_cpu_affinity() on macOS
	* Fix a misprint of extended CPUID in cpuid_basic_identify()
	* Restore previous thread CPU affinity before returning from cpuid_get_all_raw_data() (#184)
	* Query CPU info at least once even if set_cpu_affinity() fails
	* Support for AMD 19h family MSRs
	* Fix detection of Intel Core i5 Lynnfield
	* Rename set_error() to cpuid_set_error() and get_error() to cpuid_get_error() (#188)
	* Support for Intel Alder Lake-N
	* Support for AMD Rembrandt with Radeon Graphics
	* Support for Intel Raptor Lake-S with "Golden Cove" cores
	* Support for Intel Raptor Lake-P
	* Support for Intel Raptor Lake-U
	* Support for Intel Rocket Lake-E


Version 0.6.2 (2022-11-11):
	* Fix segmentation fault in cpu_identify_all() for single-core CPUs
	* Support for Intel Penryn L
	* Support for Intel Tremont
	* Support for AMD Mendocino
	* Support for Intel Ice Lake (Xeon-D)
	* Support for AMD Zen 2 Desktop Kit CPUs (4700S + 4800S)
	* Support for AMD Athlon 64 Sherman


Version 0.6.1 (2022-10-23):
	* Support for AMD Raphael
	* Support for AMD Dali
	* Support for AMD Van Gogh
	* Fix stuck cpuid_tool due to set_cpu_affinity() on Windows (NetBSD#172)
	* Remove AMD Warhol from DB (Zen3+ cancelled)
	* Fix physical core count computed by cpu_identify_all() when HT is disabled (NetBSD#175)
	* Fix shared library symlinks with CMake (NetBSD#174)
	* Support for Intel Raptor Lake-S


Version 0.6.0 (2022-09-23):
	* Support for Intel Tiger Lake
	* CPU Family/Model is used as Ext.Family/Model
	* Use popcount64 from libc when available (NetBSD#152)
	* Reduce minimum CMake requirement 3.14 to 3.13
	* Do not link with msrdriver.c on non-Windows platform (NetBSD#159)
	* Report memory allocation failures without making a segmentation fault (NetBSD#160)
	* Support for AMD Lucienne
	* Support for AMD Rembrandt
	* Support for AMD Warhol
	* Remove Debian package from source tree (NetBSD#165)
	* Fix build under Clang 15 (NetBSD#167)
	* Support for AMD Athlon Godavari
	* Support for hybrid CPU like Intel Alder Lake (NetBSD#166)
	* Detect presence of hypervisor (NetBSD#169)
	* Decode deterministic cache info for AMD CPUs (NetBSD#168)
	* Add cache instances field in cpu_id_t and system_id_t (NetBSD#168)
	* Support AMD Bald Eagle
	* Support for more AMD Godavari (Athlon)
	* Rename AMD Bulldozer to Zambezi
	* Support for AMD Interlagos
	* Support for AMD Abu Dhabi
	* Support for AMD Beema
	* Support for AMD Steppe Eagle
	* Support for more AMD Kabini (Sempron + Athlon)
	* Improve msr_serialize_raw_data()
	* Support for AMD Zen 2 custom APU for Steam Deck
msk pushed a commit to msk/pkgsrc that referenced this pull request May 11, 2026
Added
 - Delete line with Ctrl-R (NetBSD#114)
 - Copy, cut and paste lines with Ctrl-C, Ctrl-X, Ctrl-V (#207)
 - Ctrl+arrows moves to the previous/next word (#214)
 - Syntax highlighting configurations for C (NetBSD#98, #181), Nim (NetBSD#106), C# (#211), C++ (#211), CoffeeScript (#262), CSS (#211), D (#262), Dart (#211), Elixir (#211), Fish (#211), Go (#211), Groovy (#262), Haskell (#211), HTNL (#211), Java (#211), JavaScript (NetBSD#108), Julia (#262), Kotlin (#211), LRC #433, Lua (NetBSD#108, #277), Markdown (NetBSD#152), MATLAB (#262), Nix (#262), NoSQL (#211), Nushell (#262, #433), OCaml (#262), Perl (#211), PHP (#211), PowerShell (#211), Processing (#262), PRQL (#369), R (#211), Racket (#211), Ruby (#211), Raku (#262), RSS #433, Scala (#211), SQL (#211), Swift (#211), TypeScript (#211), XML (#211, #449), YAML (#211), Zig (#262, #400), ZSH (#211)
 - Support for WebAssembly (NetBSD#159)
 - Binary optimization for release: enable LTO (#346); single codegen unit, abort on panic, strip symbols (#464)
 - kibi --version includes git revision when available (NetBSD#176)
 - Extension-less dotfiles can now have a syntax highlight configuration (#449)
 - Minimum Supported Rust Version (MSRV) in Cargo.toml (NetBSD#122, NetBSD#133, NetBSD#175, #191, #306, #307, #343, #442)

Changed
 - Syntax highlighting configuration for V (NetBSD#108)
 - Use alternate screen buffer to avoid flicking, restore the terminal content on exit (#310)
 - Various no-op code changes to reduce line count (NetBSD#127, NetBSD#151, NetBSD#154, #191, #229, #280, #335, #331, #330, #422) and to fix Clippy warnings (NetBSD#175, #188, #190, #206, #241, #249, #321, #334, #345, #385)
 - Rust edition: 2024 (#442)

Fixed
 - Crash when opening a new file (#287)
 - Error message when an invalid option is provided (NetBSD#150)
 - Error message when trying to open a special file, e.g. UNIX devices or directories (NetBSD#159)
 - Config parsing with invalid durations (#340), invalid tab size (#450)
 - Emit warnings instead of panicking for invalid configurations (#449)
msk pushed a commit to msk/pkgsrc that referenced this pull request May 11, 2026
Changed templating engine

Matugen now finally uses its own custom written templating engine with using colors as the main priority.
You can import colors from json as well.

    Added

        allow output_path of template to be none
        add --import-json-string
        support multiple paths for --import-json
        add escaping for delimiters (#190)
        add string casing filters (NetBSD#157)
        improve color generation (#182)
        add back set to (closes NetBSD#177)
        allow if expressions in filter args
        add more errors for parser
        use thiserror for parser
        remove ui
        show info message when executing hooks
        add include-image-in-json (NetBSD#175)
        remove custom_keywords
        added blend. rename old blend to harmonize
        if path has a number, use a _ prefix
        add all formats for to_color
        add matugen json, change how parser resolves colors
        add blend filter
        add looping over color formats and impl display for maps
        allow expressions inside of keywords
        add binary operations (only in filter args)
        support exprs for filter args
        replace resolve_path_filter
        add range expression
        change how resolve works
        add if statement
        add include block
        add auto_lightness
        (gui) save image folder and tab
        add input_path_modes (NetBSD#142)
        (gui) add path under images
        (gui) add images tab
        add ui
        add palettes to dump_json

    Fixed

        allow escape without anything (#200)
        do not trim source for parser (#192)
        colors not saved correctly (#184)
        casting as u8 for hue (NetBSD#168)
        cache not having colors in it (#184)
        argb_from_rgb converting wrong color (NetBSD#173)
        fix minor typo in build command
        errors in included files showing span in original
        not overwriting generated files
        check metadata only if file exists
        use "choice" instead of "or" in parser
        change template adding and fix include (NetBSD#158)
        ingoring whitespace in for loop
        actually show the template name in errors
        cache not loading
        remove dbg print for path
        trying to install color eyre twice
        not compiling because of unix wallpaper
        wrong tones in dump_json (NetBSD#116)
        remove wallpaper.set option
        add dump-json to default featues

    Other

        add command.sh to example/
        add compare-script.ts
        add resize-filter arg (#182)
        remove warnings
        use wrap_err, fix read only detection
        Update main.yml
        Update main.yml
        Update main.yml
        change release-plz workflow
        move some stuff from main into helpers
        fix all warnings
        add to_color (TODO)
        remove all warnings
        move parser into its own file
        change cache colors to #RRGGBBAA
        run cargo clippy fix
        run cargo fmt
        change cache color format
        theme the ui based on image
        fix raw strings in parser
        fix ui, move cache to cache/images
        add hooks
        improve errors, add caching toggle
        add caching
        support chaning syntax
        add hsl filters, saturate
        run cargo clippy --fix, remove benches
        use IndexMap instead for ordering in for loops
        return multiple errors
        use colorsys
        replace engine, support quoted ident
        use simplespan instead
        add set_alpha
        restructure
        move stuff from engine
        run clippy fix
        remove modified_colors
        use lazy color
        remove matugen-parser
        working for loops
        kinda working for loop
        add error for arg type
        add expect_args
        rename filtertype
        cleanup imports
        remove old filters
        working no for loops
        run cargo fmt
        refactor
        (readme) improve discord link
        run cargo fmt
        (readme) add discord server link
msk pushed a commit to msk/pkgsrc that referenced this pull request May 11, 2026
1.3.0 (2025-09-23)

What's Changed

* Set required ruby version to 2.7.0 and up by @risen in NetBSD#169
* Keep using URI RFC2396 parser by @voxik in #192
* Make DEFAULT_LOCATOR Configurable by @heka1024 in NetBSD#179

New Contributors

* @risen made their first contribution in NetBSD#169
* @biow0lf made their first contribution in NetBSD#167
* @duffuniverse made their first contribution in #180
* @berkos made their first contribution in NetBSD#170
* @elia made their first contribution in #195
* @Earlopain made their first contribution in #188
* @stevenharman made their first contribution in NetBSD#173
* @voxik made their first contribution in #192
* @m-nakamura145 made their first contribution in NetBSD#175
* @heka1024 made their first contribution in NetBSD#179
* @tylerwillingham made their first contribution in #200
msk pushed a commit to msk/pkgsrc that referenced this pull request May 11, 2026
What's Changed

    feat: add just test-ci cmd to mimic ci testing locally by @anistark in NetBSD#157
    refactor: restructure sbom command with format subcommands (spdx, cyclonedx) by @anistark in NetBSD#156
    feat: validate sbom spdx and cyclonedx files by @anistark in NetBSD#162
    feat: implement comprehensive SPDX standards compliance enhancements and character validation by @anistark in NetBSD#163
    feat: add language parameter to GitHub Action by @anistark in NetBSD#164
    fix: add CC0-1.0 license to compatibility matrix for all project licenses by @anistark in NetBSD#165
    fix: exclude Go toolchain and version directives from dependency analysis by @anistark in NetBSD#166
    feat: add filter to the TUI by @ashish10677 in NetBSD#170
    chore: update dependencies by @anistark in NetBSD#168
    feat: add sort feature to TUI and proper ascending/descending order by @anistark in NetBSD#167
    refactor: move CI/CD examples to examples/ci directory by @anistark in NetBSD#171
    Refactor/publish tag by @anistark in NetBSD#172
    feat: add local-first license detection with --no-local flag for Node.js and Rust by @anistark in NetBSD#173
    feat: ignore licenses and dependencies by @anistark in NetBSD#175
msk pushed a commit to msk/pkgsrc that referenced this pull request May 11, 2026
2.098 2026/01/06
- Another fix to issue NetBSD#175 to make sure that an unblessed sockets gets
  blessed in place instead of using new_from_fd. Document that it will
  not retain original class with unblessed sockets on error, since this
  never worked anyway (there is no native unbless)
2.097 2026/01/06
- fix issue NetBSD#175 with upgrading from plain socket (no object) by using
  correct fdopen mode +< instead of <+
2.096 2026/01/04
- Allow stacking TLS layers with SSL_usebio
  This also allows LWP (after patches) to access https site through TLS
  enabled proxy
msk pushed a commit to msk/pkgsrc that referenced this pull request May 11, 2026
1.1.0 (2025-10-31)

What's Changed

* Update to use the latest version of setup-ruby and bump up to Ruby 3.4 by
  @hsbt in NetBSD#158
* Fix the mention to removed URI.escape/URI::Escape by @y-yagi in NetBSD#146
* Use a fully qualified name in warning messages by @y-yagi in NetBSD#150
* Support Ractor#value by @hsbt in NetBSD#163
* Removed unnecessary workaround by @hsbt in NetBSD#164
* Escape reserved characters in scheme name by @nobu in NetBSD#148
* [DOC] State that uri library is needed to call Kernel#URI by @nobu in NetBSD#167
* Prefer dedicated assertion methods by @nobu in NetBSD#169
* Fix the message for unexpected argument by @nobu in NetBSD#171
* Make URI::regexp schemes case sensitive (NetBSD#38) by @nobu in NetBSD#170
* The local part should not contain leading or trailing dots in the
  EMAIL_REGEXP by @nlevchuk in NetBSD#124
* More checks in EMAIL_REGEXP by @nobu in NetBSD#172
* Do not allow empty host names, as they are not allowed by RFC 3986 by
  @jeremyevans in NetBSD#116
* Improve performance of URI::MailTo::EMAIL_REGEXP by @nobu in NetBSD#173
* Performance test stability by @nobu in NetBSD#174
* Update documents that used URI::Parser by @nobu in NetBSD#175
* Add a workflow to sync commits to ruby/ruby by @k0kubun in #183
* Add irb to the Gemfile to fix the warning by @y-yagi in #182
* Replace reference to the obsolete URI.escape with
  URI::RFC2396_PARSER.escape by @vivshaw in NetBSD#166
* Switch a parsing behavior completely when switching a parser by @y-yagi in
  NetBSD#161
* improve error message by @soda92 in NetBSD#130
* Use generic version number to VERSION by @hsbt in #187

New Contributors

* @y-yagi made their first contribution in NetBSD#146
* @nlevchuk made their first contribution in NetBSD#124
* @vivshaw made their first contribution in NetBSD#166
* @soda92 made their first contribution in NetBSD#130


1.1.1 (2025-11-04)

What's Changed

* Re-allow consecutive, leading and trailing dots in EMAIL_REGEXP by @osyoyu
  in #189

New Contributors

* @osyoyu made their first contribution in #189
msk pushed a commit to msk/pkgsrc that referenced this pull request May 11, 2026
## [0.9.4] - 2026-02-03

### Major: Full Windows Support

This release adds **full native Windows support**. libfyaml now builds and runs natively on Windows with MSVC, clang-cl, and Clang compilers.

**Native Windows builds:**
- Full support for building on Windows using MSVC, clang-cl, or GCC
- Works with Visual Studio, VSCode, and other Windows development tools
- All tests pass on Windows

**Cross-compilation from Linux:**
- Support for msvc-wine to install MSVC redistributables on Linux
- Compile using `cl.exe` via Wine
- Compile using `clang-cl` without Wine
- Compile using Clang cross-compilation triplet

### Major: Comment Support Now Stable

Comment parsing and emission has been moved out of experimental status. Comments can now be reliably preserved and manipulated through the API.

### Added

- `fy_node_set_style()`: Set the style of a node (block, flow, plain, etc.) - Fixes NetBSD#78
- `fy_token_set_comment()`: Attach comments to tokens programmatically
- `fy_event_to_string()`: Convert events to string representation
- `fy_diag_get_collect_errors()`: Query if error collection is enabled
- `fy_atom_lines_containing()`: Get lines containing an atom (for diagnostics)
- `fy_memstream`: Portable `open_memstream` alternative for cross-platform support
- CMake-based CI workflow with improved matrix coverage
- Emscripten platform detection for `endian.h`

### Changed

- libclang now defaults to OFF (will be enabled when reflection features are ready)
- Document start token is now preserved (may contain comments)
- Walk methods now handle error paths more systematically
- Removed non-existent experimental function declarations from `libfyaml.h`

### Fixed

- **#193**: Token creation now properly clears memory to avoid undefined behavior on invalid input
- **#186**: Reference loop nesting now respected when checking link validity
- **#185**: Fixed crash when setting document root to NULL; also fixed input size clamping for corrupted input
- **#184**: Walk memory leak fix with improved debugging infrastructure
- **#183, #191**: Error out early on `FYECF_EXTENDED_CFG` with helper emit methods (prevents crash)
- **#182**: Walk expression unref bug fix with debug infrastructure
- **#181**: Walk double-free on node delete
- **NetBSD#178, NetBSD#177**: Walk methods now handle error paths systematically (recursive alias resolution)
- **NetBSD#176**: Off-by-one error in `fy_accel_grow`
- **NetBSD#175**: Parser crash on corrupted UTF-8 at end of file
- **NetBSD#174**: Superfluous document end marker with explicit version/tag directives
- **NetBSD#173, NetBSD#172**: Depth limit for node copy (prevents stack overflow under fuzzing)
- **NetBSD#143**: Document root now correctly marked as attached
- Emit state now properly reset at end of document (fixes multi-document stream markers)
- Flow quoting error on ANY style (test was backwards)
- Empty file `fdopen` issue on some platforms
- Empty stream `realloc(0)` undefined behavior
- Removed jarring notice when alias is declared multiple times (valid YAML)

### Platform Support

**Supported platforms**: Linux, macOS, FreeBSD, OpenBSD, NetBSD, and **Windows**.

**Windows-specific:**
- Full native MSVC support (32-bit and 64-bit)
- clang-cl and Clang cross-compilation support
- msvc-wine support for Linux-based Windows cross-compilation
- Proper CRLF (DOS line ending) handling
- Fixed 32-bit MSVC intrinsics (`_BitScanForward64`, `_BitScanReverse64`, `__popcnt64`)

**Portability fixes:**
- Fixed void pointer arithmetic (GCC extension) for strict C compliance
- Fixed GCC ternary operator extension (`x ? : y` -> `x ? x : y`)
- Fixed `\e` escape sequence (GCC/clang extension) -> `\x1b`
- Fixed enum comparison warnings across platforms
- Align mremap initial size to page boundary (fixes BSD crashes)

**macOS:**
- Fixed ASAN support (requires `-fsanitize=address` at link time)
- Added extra ASAN flags for Apple's clang (alloca poisoning disabled)

**CI/Build:**
- New CMake-based GitHub Actions workflow
- Improved build matrix coverage
- Fixed distcheck breakage

### Internal

- Walk expression debug infrastructure for easier debugging
- Portable `fy_memstream` wrapper for `open_memstream`
- Use `fy_align_alloc/free` wrappers in allocator
- Fixed allocator `get_caps` return type (enum, not int)
- Atomic counter function instead of macro
- General warning cleanup pass

### Statistics

- 58 commits since v0.9.3
- 18 bug fix issues closed
- Full Windows platform support added
msk pushed a commit to msk/pkgsrc that referenced this pull request May 11, 2026
## [v3.3.1] - 2026-02-26

### Fixed

- Fixed query selector parsing issue that incorrectly parsed array accessors when they followed a `filter` or `map` call.

## [v3.3.0] - 2026-02-25

### Added
- `replace` function to replace occurrences of a substring in a string with another string. [See docs](https://daseldocs.tomwright.me/functions/replace).

## [v3.2.3] - 2026-02-23

### Added
- XML parser now preserves comments and processing instructions during round-trip (NetBSD#175).

### Fixed
- Spread operator within array construction is now honoured.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants