I don’t consider myself exceptional in any regard, but I stumbled upon a few cryptography vulnerabilities in Matrix’s Olm library with so little effort that it was nearly accidental.
It should not be this easy to find these kind of issues in any product people purportedly rely on for private messaging, which many people evangelize incorrectly as a Signal alternative.
Later, I thought I identified an additional vulnerability that would have been much worse, but I was wrong about that one. For the sake of transparency and humility, I’ll also describe that in detail.
I don’t consider myself exceptional in any regard, but I stumbled upon a few cryptography vulnerabilities in Matrix’s Olm library with so little effort that it was nearly accidental.
It should not be this easy to find these kind of issues in any product people purportedly rely on for private messaging, which many people evangelize incorrectly as a Signal alternative.
Later, I thought I identified an additional vulnerability that would have been much worse, but I was wrong about that one. For the sake of transparency and humility, I’ll also describe that in detail.
Many people have, over the years, assumed the opposite and commented accordingly. The ensuing message board threads are usually is a waste of time and energy for everyone involved. So please adjust your expectations.
2024-05-15: I took a quick look at the Matrix source code. I identified two issues and emailed them to their security@ email address. In my email, I specify that I plan to disclose my findings publicly in 90 days (i.e. on August 14), in adherence with industry best practices for coordinated disclosure, unless they request an extension in writing.
2024-05-16: I checked something else on a whim and find a third issue, which I also email to their security@ email address.
2024-05-17: Matrix security team confirms receipt of my reports.
2024-05-17: I follow up with a suspected fourth finding–the most critical of them all. They point out that it is not actually an issue, because I overlooked an important detail in how the code is architected. Mea culpa!
2024-05-18: A friend discloses a separate finding with Matrix’s protocol. (Will update this to link to their write-up once it’s public.) They instructed the Matrix developers to consult with me if they needed cryptography guidance. I never heard from them on this externally reported issue.
2024-07-12: I shared this blog post draft with the Matrix security team while reminding them of the public disclosure date.
2024-07-31: Matrix pushes a commit that announces that libolm is deprecated.
2024-07-31: I email the Matrix security team asking if they plan to fix the reported issues (and if not, if there’s any other reason I should withhold publication).
2024-07-31: Matrix confirms they will not fix these issues (due to its now deprecated status), but ask that I withhold publication until the 14th as originally discussed.
2024-08-14: This blog post is publicly disclosed to the Internet.
As of 2009, you could remotely detect a timing difference of about 15 nanosecond over the Internet with under 50,000 samples. Side-channel exploits are generally statistical in nature, so such a sample size is generally not a significant mitigation.
How is this code actually vulnerable?
In the above code snippet, the vulnerability occurs in aes_sbox[/* ... */][/* ... */].
Due to the details of how the AES block cipher works, the input variable (word) is a sensitive value.
Software written this way allows attackers to detect whether or not a specific value was present in one of the processor’s caches.
To state the obvious: Cache hits are faster than cache misses. This creates an observable timing difference.
Such a timing leak allows the attacker to learn the value that was actually stored in said cache. You can directly learn this from other processes on the same hardware, but it’s also observable over the Internet (with some jitter) through the normal operation of vulnerable software.
The correct way to solve this problem is to use hardware accelerated AES, which uses distinct processor features to implement the AES round function and side-steps any cache-timing shenanigans with the S-box.
Not only is this more secure, but it’s faster and uses less energy too!
Signature malleability usually isn’t a big deal for most protocols, until suddenly you discover a use case where it is. If it matters, that usually that means you’re doing something with cryptocurrency.
Briefly, if your signatures are malleable, that means you can take an existing valid signature for a given message and public key, and generate a second valid signature for the same message. The utility of this flexibility is limited, and the impact depends a lot on how you’re using signatures and what properties you hope to get out of them.
For ECDSA, this means that for a given signature , a second signature is also possible (where is the order of the elliptic curve group you’re working with).
Matrix uses Ed25519, whose malleability is demonstrated between and .
This is trivially possible because S is implicitly reduced modulo the order of the curve, , which is a 253-bit number (0x1000000000000000000000000000000014def9dea2f79cd65812631a5cf5d3ed) and S is encoded as a 256-bit number.
The Ed25519 library used within Olm does not ensure that , thus signatures are malleable. You can verify this yourself by looking at the Ed25519 verification code.
Any C library that handles base64 codecs for private key material should use a similar implementation. It’s fine to have a faster base64 implementation for non-secret data.
Most Matrix clients that still depended on libolm should treat this blog as public 0day, unless the Matrix security team already notified you about these issues.
Background Information
If you’re curious about the backstory and context of these findings, read on.
Otherwise, feel free to skip this section. It’s not pertinent to most audiences. The people that need to read it already know who they are.
End-to-end encryption is one of the topics within cryptography that I find myself often writing about.
Earlier this year, the Telegram CEO started fearmongering about Signal with assistance from Elon Musk, so I wrote a blog post urging the furry fandom to move away from Telegram and start using Signal more. As I had demonstrated years prior, I was familiar with Signal’s code and felt it was a good recommendation for security purposes (even if its user experience needs significant work).
I thought that would be a nice, self-contained blog post. Some might listen, most would ignore it, but I could move on with my life.
An overwhelming number of people took it upon themselves to recommend or inquire about Matrix, which prompted me to hastily scribble down my opinion on Matrix so that I might copy/paste a link around and save myself a lot of headache.
Just when I thought the firehose was manageable and I could move onto other topics, one of the Matrix developers responded to my opinion post.
Thus, I decided to briefly look at their source code and see if any major or obvious cryptography issues would fall out of a shallow visual scan.
Since you’re reading this post, you already know how that ended.
The bugs described above are the sort of thing I mentally scan for when I first look at a project just to get a feel for the maturity of the codebase. I do this with the expectation (hope, really) of not finding anything at all.
(If you want two specific projects that I’ve subjected to a similar treatment, and failed to discover anything interesting in: Signal and WireGuard. These two set the bar for cryptographic designs.)
It’s absolutely bonkers that an AES cache timing vulnerability was present in their code in 2024.
It’s even worse when you remember that I was inundated with Matrix evangelism in response to recommending furries use Signal.
I’m a little outraged because of how irresponsible this is, in context.
It’s so bad that I didn’t even need to clone their git repository, let alone run basic static analysis tools locally.
So if you take nothing else away from this blog post, let it be this:
There is roughly a 0% chance that I got extremely lucky in my mental grep and found the only cryptography implementation flaws in their source code. I barely tried at all and found these issues.
I would bet money on there being more bugs or design flaws that I didn’t find, because this discovery was the result of an extremely half-assed effort to blow off steam.
Wasn’t libolm deprecated in May 2022?
The Matrix developers like to insist that their new Rust hotness “vodozemac” is what people should be using today.
I haven’t looked at vodozemac at all, but let’s pretend, for the sake of argument, that its cryptography is actually secure.
(This is very likely if they turn out to be using RustCrypto for their primitives, but I don’t have the time or energy for that nerd snipe, so I’m not going to look. Least Authority did audit their Rust library, for what it’s worth, and Least Authority isn’t clownshoes.)
It’s been more than 2 years since they released vodozemac. What does the ecosystem penetration for this new library look like, in practice?
A quick survey of the various Matrix clients on GitHub says that libolm is still the most widely used cryptography implementation in the Matrix ecosystem (as of this writing):
2 of the 16 clients surveyed use the new vodozemac library. 11 still use libolm, and 3 don’t appear to implement end-to-end encryption at all.
If we only focus on clients that support E2EE, vodozemac has successfully been adopted by 15% of the open source Matrix clients on GitHub.
I deliberately excluded any repositories that were archived or clearly marked as “old” or “legacy” software, because including those would artificially inflate the representation of libolm. It would make for a more compelling narrative to do so, but I’m not trying to be persuasive here.
Deprecation policies are a beautiful lie. The impact of a vulnerability in Olm or Megolm is still far-reaching, and should be taken seriously by the Matrix community.
Worth calling out: this quick survey, which is based on a GitHub Topic, certainly misses other implementations. Both FluffyChat and Cinny, which were not tagged with this GitHub Topic, depend a language-specific Olm binding.
These bindings in turn wrap libolm rather than the Rust replacement, vodozemac.
But the official clients…
I thought the whole point of choosing Matrix over something like Signal is to be federated, and run your own third-party clients?
If we’re going to insist that everyone should be using Element if they want to be secure, that defeats the entire marketing point about third-party clients that Matrix evangelists cite when they decry Signal’s centralization.
As I mentioned in the timeline at the top, I thought I found a fourth issue with Matrix’s codebase. Had I been correct, this would have been a critical severity finding that the entire Matrix ecosystem would need to melt down to remediate.
Fortunately for everyone, I made a mistake, and there is no fourth vulnerability after all.
However, I thought it would be interesting to write about what I thought I found, the impact it would have had if it were real, and why I believed it to be an issue.
The highlighted segment is doing pointer arithmetic. This means it’s reading 32 bytes, starting from the 32nd byte in private_key.
What’s actually happening here is: private_key is the SHA512 hash of a 256-bit seed. If you look at the function prototype, you’ll notice that public_key is a separate input.
Virtually every other Ed25519 implementation I’ve ever looked at before expected users to provide a 32 byte seed followed by the public key as a single input.
This led me to believe that this private_key + 32 pointer arithmetic was actually using the public key for calculating r.
The variable r (not to be confused with big R) generated via the first SHA512 is the nonce for a given signature, it must remain secret for Ed25519 to remain secure.
If r is known to an attacker, you can do some arithmetic to recover the secret key from a single signature.
Because I had mistakenly believed that r was calculated from the SHA512 of only public inputs (the public key and message), which I must emphasize isn’t correct, I had falsely concluded that any previously intercepted signature could be used to steal user’s private keys.
But because private_key was actually the full SHA512 hash of the seed, rather than the seed concatenated with the public key, this pointer arithmetic did NOT use the public key for the calculation of r, so this vulnerability does not exist.
If the code did what I thought it did, however, this would have been a complete fucking disaster for the Matrix ecosystem. Any previously intercepted message would have allowed an attacker to recover a user’s secret key and impersonate them. It wouldn’t be enough to fix the code; every key in the ecosystem would need to be revoked and rotated.
Whew!
I’m happy to be wrong about this one, because that outcome is a headache nobody wants.
So no action is needed, right?
Well, maybe.
Matrix’s library was not vulnerable, but I honestly wouldn’t put it past software developers at large to somehow, somewhere, use the public key (rather than a secret value) to calculate the EdDSA signature nonces as described in the previous section.
To that end, I would like to propose a test vector be added to the Wycheproof test suite to catch any EdDSA implementation that misuses the public key in this way.
Then, if someone else screws up their Ed25519 implementation in the exact way I thought Matrix was, the Wycheproof tests will catch it.
For example, here’s a vulnerable test input for Ed25519:
A similar test vector would also be worth creating for Ed448, but the only real users of Ed448 were the authors of the xz backdoor, so I didn’t bother with that.
(None of the Project Wycheproof maintainers knew this suggestion is coming, by the way, because I was respecting the terms of the coordinated disclosure.)
Closing Thoughts
Despite finding cryptography implementation flaws in Matric’s Olm library, my personal opinion on Matrix remains largely unchanged from 2022. I had already assumed it would not meet my bar for security.
Cryptography engineering is difficult because the vulnerabilities you’re usually dealing with are extremely subtle. (Here’s an unrelated example if you’re not convinced of this general observation.) As SwiftOnSecurity once wrote:
The people that developed Olm and Megolm has not proven themselves ready to build a Signal competitor. In balance, most teams are not qualified to do so.
I really wish the Matrix evangelists would accept this and stop trying to cram Matrix down other people’s throats when they’re talking about problems with other platforms entirely.
More important for the communities of messaging apps:
You don’t need to be a Signal competitor. Having E2EE is a good thing on its own merits, and really should be table stakes for any social application in 2024.
My work-in-progress proposal to bring end-to-end encryption to the Fediverse doesn’t aim to compete with Signal. It’s just meant to improve privacy, which is a good thing to do on its own merits.
If I never hear Matrix evangelism again after today, it would be far too soon.
If anyone feels like I’m picking on Matrix, don’t worry: I have far worse things to say about Telegram, Threema, XMPP+OMEMO, Tox, and a myriad other projects that are hungry for Signal’s market share but don’t measure up from a cryptographic security perspective.
If Signal fucked up as bad as these projects, my criticism of Signal would be equally harsh. (And remember, I have looked at Signal before.)
But then again, poetry can be sexual, so who knows?
Scandalous furry, Why are you glitching like that? Haiku are lewd too! Art: AJ
The cowardice comes in with the fear of their peers or bosses judging them for *checks notes* the content and presentation that I wrote, and not them.
Which (if you think about it for any significant length of time) implies that they’re generally eager to take credit for other people’s work, but their selfishness was thwarted by a cute cartoon dhole doing something totally innocent.
Last month, I wrote a blog post about Aural Alliance, which caused a menace in the furry music space to accuse me of “bad journalism” for not verbally crucifying the label’s creator (a good friend of mine) for having a failed business venture in the past, or taking credit for donating to their cause early on.
Twitter DM conversation. Everyone I’ve talked to that has dealt with this particular person before responded with, “Yeah, this is typical Cassidy behavior.”
To which one must wonder, “Since when am I a journalist?”
I’ve never called myself a journalist. I’m a blogger and I don’t pretend to be anything more than that. I especially would never besmirch the work of real journalists by comparing it with my musings.
At times, I also wear the security researcher hat, but you’ll only hear about it when I’m publishing a vulnerability.
This is a personal blog. I will neither be censored nor subject to compelled speech. I have no moral or professional obligations to “both sides” of what amounts to a nontroversy.
Sure, I contributed to covering Aural Alliance’s up-front infrastructure costs when it was just an idea in Finn’s head. I’m not going to apologize for supporting artists. The Furry Fandom wouldn’t exist without artists.
It bewilders me every time someone reacts this way. Do you not know the community you’re in?
The most intelligible pushback I’ve seen over the years is, “Well if everyone raises their prices, low-income furries will be pushed out of the market!”
Setting aside that art is a luxury, not a need for a moment, that’s not actually true.
There are so many artists, and they’re so decentralized, that no coherent price coordination effort is even possible. It’s worse than herding cats. Some may raise their prices by $5, others by $500. If furries were organized enough to coordinate something like this, then we’d have a tough time explaining why there are still abusers in the fandom.
Also, it costs very little to learn to draw, yourself:
The demand for low-priced digital art incentivizes people to reach for theft enabled by large-scale computing (a.k.a. “AI” by its proponents).
A similar demand for cheap, high-quality fursuits (usually at the maker’s expense) will lead to a walmartization of the furry community.
If you listen to these hot takes long enough, you start to notice a pattern of short-sighted selfishness.
When you demand something of the furry community, and don’t think of the long-term consequences of your demands, you’re probably being an idiot. This is true even if it’s actually a good idea.
If me supporting artists somehow prices you out of commissioning your favorite artist, you still have other options: Learning to make your own, finding new artists, saving money, etc.
On the flipside, the artists you admire will suffer less due to money troubles. Fewer artists starving makes the world a more beautiful place.
Center of the Fediverse
If flame war and retoot count relieved desire In the comment thread someone must have known That the hottest takes truly leave us tired ‘Cause in the center of the fediverse We are all alone
If you’re on the Fediverse (e.g., Mastodon), and your instance uses a blocklist like TheBadSpace (TBS), you probably cannot see my posts on furry.engineer anymore.
(Spoiler: It was largely prompted by another predominantly LGBTQIA+ instance, tech.lgbt, being erroneously added to the same blocklist, which resulted in criticism of said blocklist curators.)
Be forewarned, though: Linking to Silver Eagle’s blog post was enough for TBS supporters to harass me and directly accuse me, personally, of anti-blackness, so don’t expect any degree of level-headed discussion from that crowd.
If you cannot see my Fediverse posts anymore, and actually want to see them, message your instance moderators and suggest unsubscribing from TheBadSpace’s blocklist.
If they refuse, your only real recourse is to move to another instance. The great thing about the Fediverse is, you can just do that, and nobody can lock you in.
Personally, I plan on sticking on furry.engineer. I trust its moderators to not tolerate racist and/or fascist bullshit.
The baseless accusations of anti-blackness are, unsurprisingly, false.
Burnout Isn’t Inevitable
A few months ago, I quit a great job with an amazing team because the CEO decided that everyone has to return to working in the office, including people that were hired fully remote before the pandemic. This meant being forced to move more than 3,000 miles, or resigning. I’ve been told the legal term for such a move is “constructive dismissal.”
In hindsight, I was starting to burn out anyway, so leaving when I did was a great move for my mental health and life satisfaction.
I’m an introvert. I have a finite social battery. Because my work was split across three different teams at the same company, I was a necessary participant in a lot of meetings.
More than 5 hours per day of meetings, as an individual contributor. Sometimes as many as 7 hours/day of them. I almost never had a quiet day, even after blocking one day every week so nobody would schedule any meetings and I could get productive work done.
If you’re interested in being a people manager, or have an extroverted personality, you’re probably unperturbed by this account. But I was absolutely miserable. My close friends started to worry if I was suffering from depression, because of how socially exhausted I was all the time.
I took a few weeks off between jobs. My new role doesn’t pointlessly encumber me with unnecessary meetings.
Every day, I feel the burnout symptoms leaving my mind. I feel challenged and stimulated in a good way. I’m learning new technologies and being productive. I’ve never spent more than 3 hours of any given day in a meeting.
Different people burn out in many different ways, for many different reasons.
In my experience, the consequences appear to be reversible if caught early enough. I don’t know if they would be if I held onto my old job for much longer.
The job market’s tough right now, but if you’re deeply unsatisfied with an aspect of your current job, prioritize yourself and make whatever change is necessary.
This doesn’t mean you have to switch jobs like I did, of course. It was a good move for me. Your mileage may vary.
This post has been a collection of unrelated topics on my mind over the past few months. There is one other thing, but I was unsure if it warranted a separate post of its own, or an addendum on this one. Since you’re reading this, you’ll know I ultimately settled on the latter.
I started this blog in 2020 because I thought having a personal blog where I talk about things that interest me (mainly the furry fandom and software security) would be fun. And I wanted to do it in a way that was fun for me.
“Having fun with it” has been the guiding principle of this blog for over 3 years. I never intended to do anything important or meaningful, that sort of happened by accident. I didn’t care about others being able to use my writing in a professional setting (hence, my scoffing at the very notion above).
Lately, posts have slowed to a crawl, because it’s not fun for me anymore. I have a lot of ideas I’d love to write about, but when it comes time to turn an idea into something tangible, I lose all inspiration.
So I’m not going to force it.
This will be the last post on this blog for a while. Maybe forever, if I don’t feel like coming back. I recently tried to pick up fiction writing, but I’m not happy with anything I’ve been able to produce yet, so I won’t bore anyone with that garbage.
There are a lot of brilliant people that read my writing. Most of you are more than capable of picking up where I left off and starting your own blogs.
I encourage you to do so.
Have fun with it, too. Just remember, when it’s time to put the pen down and take a rest, don’t be stubborn and burn yourself out.
Several people have reported experiencing eye pain, vision problems, and sunburnt skin on Sunday after attending ApeFest, a Bored Ape Yacht Club NFT collection event in Hong Kong.
Art by Harubaki
Credit: Tirrelous
Twitter DM conversation.
Art: CMYKat
Art: CMYKat
Art: CMYKat
