The matrix-rust-sdk
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Sparber, Julian
Jelić, Damir
Matrix is an open protocol for secure, decentralised communication - defining an end-to-end-encrypted real-time communication layer for the open Web. Historically the network has been made up of newly written native Matrix clients, or bridges to 3rd party existing chat systems (e.g. Slack, Discord, Telegram). The matrix-rust-sdk is a modular Matrix client library, meant to be a robust implementation of the protocol, and to make even the most advanced features such as E2EE easy to use. This talk will walk you through the design decisions and the tradeoffs that come with it, give you an overview of where we're at and where we're going with Web-Assembly, and finally what the future holds for the matrix-rust-sdk. Finally it will be connected to a real-life example of software using the matrix-rust-sdk: Fractal-Next.
 Hello, Faslem. Welcome to my quick talk about the Matrix Rust SDK. The Matrix Rust SDK is the Matrix Client Library. This talk will be in two parts. I'm first going to talk about the Rust SDK, how the SDK came to be, what's designed, a case study, how it's used in Android land. And after that, we're switching to Julian's father, who will talk about Fructal, a GTK Matrix Client written in Rust, which heavily uses the Rust SDK. Before I start, a little bit about me. I'm Damir. I'm rural Croatia, and since there's nothing to do in rural Croatia, I don't have some alcoholic or thanks to the advances in technology and work. So I thought to myself, why not do both? Before we jump directly to the Rust SDK, let's first talk about history a bit. How did we end up with the Rust SDK after all? Thanks for starting when I was shopping around for a bit to chat with close friends in a secure manner. I was already using BeatShot for ISC and XMPP, and I didn't want to change my work. The classic XKCD, don't change our man's workflow, if you remember that one. The one could use off the record messaging with XMPP, but that turned out to be a bit complex for non-tech friends. For those who don't know what BeatShot is, it's a text user interface ISC Client written in C and some 18 years old. It's written in this modular way that cross essentially a chat-specific text user interface framework, and everything else is a plugin. The ISC support is a plugin as well. You can expand the client to support any type of protocol. Now there are a bunch of assumptions that the text user interface has that are ISC-specific, so you might sooner or later run into trouble with some different protocols. Here's what we are dealing with. It's a single-fet it program. It's event loop-based and call-back-based. Extensions can be written in a bunch of languages. It can't spawn threads unless you're writing your extension in C. It can open sockets for you. It's important it's opening sockets in a separate thread to not block the UI thread. I have to mention that there already was support from Matrix implemented as a new extension, though the end-to-end encryption support became stale. It didn't support the latest version and has had various other problems. I started writing a new extension in Python, and I was a bit annoyed that I couldn't use at the time existing Python SDK. This nudged me towards making sure that whatever I write for Richard Matrix should be reusable for other projects and plans. Therefore, other featured Matrix came new, a modular Python client lab with a no-year API, a very push-pull HTTP request in and out of state machine and an async API on top of that. Sadly, there are still problems with this, at least for the Richard use case. End-to-end encryption support in Matrix requires sending out multiple requests in order. That quickly leads to callback hell if you're not using async arrays like we can't use async arrays and reach it. Another problem is that there's no multiple threading, and expensive cryptographic operations might block the UI. If you're sending an encrypted message, the encryption keys need to be sent out to a bunch of devices. You need to encrypt for each device separately, and this might take some time, depending on the size of the room. And in that meantime, the UI will block. So relatively early on, I started to experiment with Rust bindings for the Richard plugin API and a accompanying Rust Matrix client library. This leads us now to the Rust SDK. It also has modular structure. These names are all trade names except the state cell. This one should become in the future a separate trade as well. You could pick any of these trades and integrate it into an existing client library, possibly in another language, or you could pick the whole Matrix SDK trade and blend it into another language. I should note that we are using Vroomer, so input validation layer. It passes all the responses into high level types, so we don't need to deal with JSON directly. So the Matrix SDK trade here should be the most interesting one for library developers. We are developing a Matrix client library, which doesn't yet support end-to-end encryption. Check out the crypto trade. As a simple push-call API, you pull HTTP requests out of the lib and push HTTP responses into the lib. You don't need to understand the complicated logic of how these responses need to be handled or passed or which requests need to be sent out when and how to verify signatures, signed objects, all of that is handled transparently in the crypto trade. As for the Matrix SDK trade, if you remember the code sample from Neo, we mostly retained a relatively simple Python API. Most things should be quite straightforward and end-to-end encryption support should be completely transparent. Messages get encrypted automatically and possible decrypted as well. Now this is really similar to Neo. We are using more types obviously because we are in Rustland, but all the flow is kept really similar. If you are familiar with Neo, this should be an easy upgrade there. But also if you are not familiar with Neo, it should be relatively simple to write things. Of course, if you are interested in writing bots, this might be a little bit low-level for you. If you are interested in a higher-level bot API on top of the Rust SDK trade, it would be the way to go. A bit more about the data flow in these trades. I already mentioned briefly that we have this proofable design. At the top here lies the home server, which has data. The Matrix SDK trade does not sync HTTP requests, which as for data, this request will contain an initial state. Some sequent calls will contain only the delta changes from the previous state. We push the response into the Matrix SDK base trade and into the Matrix SDK crypto trade. Those two trades contain state machines that consume the HTTP responses, update the internal state and finally generate some outgoing requests. Those requests are then important to keep the encryption going or to keep your state correct. You need to track other people devices periodically, upload new cryptographic keys to the server. Basically everything that needs to be sent to the server to have working end-to-end encryption support will be pushed out of the state machine as a group request that needs to be sent to the server. This does mean that all our networking IOS is having in the Matrix SDK trade, while base and crypto trades behave mostly as pure state machines. Some persistence is there, like data we are possessing encryption keys and to a state and so IOS is not pulled out of the trades but is abstracted away as a trade. Whereas the Matrix SDK used, other dimensions, fractal, this is a high-level, probably flagship plan that is using the SDK. It supports encryption, it supports QR code verification, emoji verification. Every advanced feature should be supported by fractal. Richard Matrix is my rewrite of Richard Matrix. It has become a bit stale due to lack of time but eventually it should replace Richard Matrix. We have the Matrix SDK app service crate. This one is built on top of the SDK and it's used to build ridges and privilege bots for various use cases. Some things only use the crypto crate. Element Android, for example, uses the Matrix SDK crypto crate and the Matrix Bust SDK as well uses the crypto crate. Element Android is written in Kotlin and Matrix Spot SDK is written in JavaScript and there are bindings from Rust to Kotlin and JavaScript. Let's take a closer look into how the Rust SDK is used in Element Android. Do note that this isn't yet in production but in a late beta phase. It only uses the crypto crate and we will take a look at how this is integrated. Firstly, we are using Mozilla's UniFFI crate to mostly generate Kotlin bindings. This is a bit different if you are familiar with Buzz and Buncheon and PiO3. It's a bit different from those crates. You have a separate file where you define your interface and then it generates Kotlin bindings or JavaScript bindings or Python and Ruby as well bindings from the separate definition. Our mind structure here is called the All Machine. This is the state machine that handles the cryptography and let's take a look at the code snippet. This is the first method that we need to bind. This is skipping the All Machine initialization but creating all machine is quite simple. We just pass a user ID and device ID and if you remember our data flow graph, this method is the push method. It pushes changes we receive from the server into the state machine. You don't need to push the whole sync response into the state machine. You just push the smaller parts into it. Those are all directly tied to the crypto support. The two device messages will contain room keys, the device changes are important to track which devices a user has and the key counts are important so we know which keys we need to replenish from the server. The two device events here will then be decrypted and returned by the All Machine and you can then consume those events separately. Next up is a pull method. This pulls all the HTTP requests out that are either here because we first initialize the All Machine or the sync changes produced. There are a bunch of request types we will get from this. I have not listed them all. It's best to send them all out in parallel and you need to notify the All Machine after each request is sent out. The response needs to be passed back to the All Machine so it knows you sent out this request. One of the requests fails to be sent out which can happen especially on mobile connections. SDK will keep answering you with the request until you tell that this has been sent out. This is a common pitfall with various clients. People tend to forget to try sending out a request and that request might have sent the encryption keys to other people and suddenly people can encrypt the messages. As a backup plan, if you fail to retrieve a request sending, the All Machine will have your back and let you know you need to send this out. This isn't done for all request types. Some of them are of memorial after all but the most important ones for example the Groom keys, the decryption keys will be kept in memory until you send them out. That's it. Now you can decrypt the messages. This doesn't happen automatically. We have a separate method for this because we don't know what your user is looking at and you want to prioritize decrypting the messages that your user is looking at. So you can do that quite easily. Now for the sending side, we need to do two things. We need to establish one-to-one secure channels between each device we're going to talk to and we need to send each of these devices a room key encrypted using the secure channel. So you first ask the All Machine, hey, who doesn't have this secure session? And it returns a request and after you pass the response to the All Machine, mostly everyone will have the secure channel established. After that, we call the share room key method and this will encrypt the room key for this given room and for those members and this will generate either one request or a bunch of requests depending how large the room is and you can send those out in parallel as well. And if you fail to send some of these, the All Machine will tell you, hey, you need to send this out the next time you call a share room key method. And that's it. Now you can encrypt messages as well. Well, almost. I skipped one method where you need to tell the SDK which users are part of encrypted rooms so it can track devices users have. But even so, I think this is quite a simple API compared to what you need to do if you implement this from scratch. Let's take a quick look at the performance of Element Android using the Matrix SDK Crypto trade, if until now nobody has been convinced to use the Rust SDK Crypto trade, perhaps this will convince them. This has been already explained to reiterate and rate the client to send an encrypted message if they just need to share what we call a room key with every device in the room. To do this, a room key needs to be encrypted for each device separately. And this graph shows how long clients take to share this room key. Now this isn't an apples to apples comparison since we use different devices, but the slowest device should definitely be the Android mobile phone. And this is the mobile phone, Android mobile phone is the one that's just using the Rust SDK. So here the corroded and rates on the graph are Element Android converted to use the Crypto trade of the Rust SDK. So we managed to lower the time required for minutes for seconds. Now this isn't all because it's Rust or because we're using multi-threading. There are certainly problems in Element Android, but I don't think anybody uses multi-threading to encrypt room keys. And therefore I don't think anybody comes close to the performance of the Rust SDK. And sharp-eyed people might notice that there are two versions of the corroded Android code base. One is neatly called Wothers Emmets. And we'll get to that next. What does this mean? So a bit of a discretion here and what does Wothers Emmets actually? Wothers Emmets is the craving name of amphibians. Why amphibians? Well, amphibians have some amazing self-healing properties. Like a gecko, you catch the tail of a gecko, the gecko will discard the tail and it will regrow. And because the double-red should protocol, which is used in Matrix, has amazing self-healing properties as well. Implementations were usually named as some amphibian. The original implementation for TechSecure, the predecessor of Signal, was called Exolotl. The Matrix version was called all or is called all. And Wothers Emmets is a pure Rust implementation of the Matrix variant of the double-red for Tupel. So why would you ever rewrite your cryptographic library? Why would you ever rewrite anything in Rust? Yes, LeBolm was stable and widely used. But let's take a look at the API of LeBolm and see why we might not like it that much. This snippet creates an ALMA account. This is basically the identity of your device. Each one of your devices will have its own ALMA account. So this involves a couple of steps. Let's ask LeBolm how many bytes we need to allocate. Ask LeBolm how many bytes of randomness we need to generate, allocate the memory and generate the randomness, and then call the initialization method. So this is a five-step process. And basically, anything you do will have the same layout here. And this shows us a couple of problems with LeBolm. It has a vitri-type API. You pass in a bunch of opaque buffers. If you reuse your buffers, you might get a slightly incorrect result. That's hard to notice. So you have your input buffer, you have your randomness buffer, you have your output buffer, and all of those could be interchanged without anybody noticing. This has actually happened to us where we got a slightly incorrect base 64 encoded message. So the input buffer was reused as the output buffer. So the base 64 encoder was encoding an already encoded thing. So it was slightly incorrect and hard to reproduce a bug in another library. So all of that happened because buffers are all opaque and you don't know what you're passing in. And the API is quite complicated. You need to make multiple calls to generate randomness, allocate things, and every language binding of the library will need to again allocate things and generate randomness separately. And the next one here is scratch-dangerous. As you find allocation size might return negative value. Every time you want to do something, you ask the library how many bytes you need to allocate. And there are a bunch of such methods. Some of them might return on error, some of them can't. If you forget to handle this error, which will be returned as a negative value, you might try to allocate all of the bytes because the negative value will be truncated to a very large value. And you possibly just edit denial of service into vulnerability into your client. Furthermore, the underlying libraries that are used for the cryptographic primitives crypt255.9, ed255.9, and encryption standard run that create either. So these are just some problems of LeBone that it shows already that people need to work a lot of LeBone to get it to a modern state. So let's take a look at what does MADS API that creates an account. These snippets are three different languages and all of them create an account in the same way. You just call the constructor and everything gets allocated and randomness gets generated. This isn't really a criticism of C or LeBone. At the time, it was probably the best idea to leave their allocations to various platforms. But nowadays, the tooling will take care to use the correct implementation on the VASM and Python and Rust. And this looks much simpler, right? Bindings become much easier and can have the same high-level API as the Rust version. The bindings also don't allocate or generate randomness anymore. So here's what the MADS offers. MADS high-level API. Thanks to Rust, it's easily bound into different languages. Using VASM, VIN, and BioFree, you name your language binding straight. We are using modern implementations of cryptographic primitives from Rust, crypto, and from the Dalek traits, so X25519 Dalek and DD25519 Dalek. And all of those have been audited. We are also going to audit for the Zemmats. And of course, we provide an upgrade app from LeBone. So LeBone has a way to persist all the objected traits. And for the Zemmats, you can read the data format LeBone uses. We are still not sure what data format for the Zemmats will use. Maybe a server-based or maybe it will use the binary data format LeBone uses. And the API is quite similar to what different on-bindings were exposing, so it shouldn't take too long to switch to all those Zemmats. And this concludes the brief overview into the Rust SDK and what else around it is happening in. The rest of the talk will take a deeper look into Fractal, how it works, and how it utilizes the Rust SDK. And this part will be explained by Julian Faber as mentioned, and off to Julian. Bye. Hi, all. I'm Julian. I'm going to talk about Fractal today and the journey of writing it. So Fractal is a native matrix client built for the GNOME desktop. So it follows the design principle laid out by the GNOME design team. It works closely with them. And therefore, it has a huge focus on usability as well as on user experience. So before we dive into the project, let's have a quick look at the project's history. So it was startled all the way back in 2017 by Daniel Brasio Moreno, who wanted to build a GTK client for Rust. And therefore, he forked Rumor GTK, which now is called Festa I Am, written by Jonas Blathe, even though it did not see much development since the fork, actually, or in the last couple of years. But Fractal, on the other hand, got huge attention over the last years. We even participated multiple times in GSOC and Outreachy. And I have been one of the GSOC students and later than also as Manchur, which was quite fun. And yeah, now five years. So it has been a long time. But now what we are going to do is Fractal complete. I can answer that quickly. No, it's not. We're actually doing something interesting. Back in February, so a year ago, obviously a year ago, we decided to do a rewrite. A rewrite? Yes, a rewrite. But this time with N2N encryption support. Fractal, even though we wanted it from the very beginning, didn't ever have encryption support because Fractal started out as a pre-time project. And it has been most of the time run by or developed by volunteers. So it was really hard to get something like encryption support built in your pre-time, actually. So we now got around. We keep moving it. And yeah, but this time it's going to be different. We did it from the beginning and now I'm going to tell you a couple of reasons why we decided to do the rewrite. So first of all, probably new. Already it's made with Rust SDK. It's pretty new. It's another version. But it's very usable and we also spent a lot of time to contribute it and provide feedback to it so we could use it. And we managed to build a client on top of it, which is not finished yet, but I will show it later when we start. The next reason is GTK4 got released a year ago, which is the UI toolkit we use to build Fractal. Initially we used GTK3 because that was the most recent version we could use. And now we have GTK4, which comes with a lot of improvements, as well as the bindings to Rust because we are talking about Rust Word. GTK is written in C. It got huge improvement, the so-called bindings, language bindings, which connect GTK4 into the Rust Word. And all the people, a shout out to them, they did an awesome job improving it and working on it the last few years, probably for quite some time. And now it's changed completely. Therefore our code base was pretty much outdated because we could use all the new fancy features of GTK4 and new bindings for it. Last but not least, LibAdvaita was released. Quite recently, the stable release was released quite recently, like two months ago or three months ago. And that's library, the successor of LibHandy. Maybe you heard of it already. It's library to build responsive GTK apps, adaptive GTK apps. So Fractal will run also on small screen sizes, for example, mobile phones. And actually LibAdvaita is much more than just a library to build adaptive apps, but also a building block for every modern application. So all of that sounds like it would reason, right? And yes, you are right. It is. So after first trying to integrate the major Rust SDK into mostly by Hando, who also was a GSOC student, into the old Fractal code base, we realized quickly that it's not going to work because so much things changed. So then in February, as I am a couple of times already said, we launched the initiative Fractal Next. So now it's all about rewriting the Fractal in native metrics time. So at this point, I need to mention that this effort would not be possible in my free time or that matter for anybody, probably, to build a project in that time span with that many hours working on it if it was not financed by somebody. We are lucky. I applied last year for funding from an internet and we want to grant as part of the private and trust enhancing technologies fund to implement and to enter encryption in Fractal. So the initial plan was always to use the matrix SDK and build on top of that, but the idea was not to rewrite Fractal initially. But after a couple of months, we pretty soon realized that it does not make sense to spend more time on the current code base or back then the old code base. Therefore, we asked for an extension and they were happily happy to support us also by, but they rewrite the accomplished out. And an internet gets most of the money from the EU Commission pretty much directly, so it's sponsored by the EU Commission, you can say. Additionally, I want to thank Kevin, Tobias, Kajar, and Alejandro for having a lot during the last few months, Kajar and Alejandro mostly during the summer months, Kaj the other time, Tobias with design support during the entire project and it's still not finished, just to mention that again. So now let's get a little bit more technical and see what's happening inside Fractal. So as you probably heard in the previous part of this talk, the Rust SDK is multi-freted and uses as an asynchronous framework Tokyo, not early, but mainly that one. On the other side, Fractal uses GTK to draw widgets and UI, which requires it to be single fretted. Obviously, this does not mean that Fractal can use multiple frets and we also use multiple frets. So the idea we developed is that we have a main frett called UI frett that handles all the GTK things. And on the other side, we have Tokyo running tasks needed for the Rust SDK, like making requests and everything the Rust SDK does for us. And to communicate between those two, we use different approaches, or actually two different approaches. Mostly, we use channels and features, which are if you're familiar with Rust or in general any modern language, you probably know them already, they allow you, the ways or channels are ways to communicate between frets or tasks, which we do in the case of Fractal. Furthermore, we need to wrap all the Rust trucks and matrix events, matrix trucks, the matrix Rust trucks into the object so that we can have a data model which can then be bound to the UI, which we have already. Additionally, the matrix Rust SDK provides mostly an asynchronous API, which does not mean that data is not immediately available, but we can rely on that because it may be on a remote server and the matrix Rust SDK still needs to request that information. Therefore, we do some local caching inside the G object, which is pretty much an object holding data, which then on the other hand provides us with a synchronous access to the data so that the UI feels snappy and reactive. So, matrix, one of the most important building blocks as the sync request, and also in Rust SDK, you need to prepare the sync to synchronize the state between the home server and your local machine. Most of that is handled actually by the Rust SDK, which is pretty nice, but there are different ways to obtain the sync response from the SDK. One is to manually make the sync request and do the SDK handle anything, which obviously you probably don't want because then you have to handle everything yourself, like encryption, key sharing and that stuff. Then there is another version where you get a stream, so it's a stream of sync responses and you can read from the stream and the matrix SDK will push new responses it requests from the home server to the stream. Maybe we are using that version and move it to a channel to be able to update our G objects, so our local cache of the state of your client or for example, a root stage or something like that. The third option, which is quite new and we would like to switch to that sooner or later, but we did not get around is that you can register event handlers, so you can say something like or write something like I want to get notified when a certain notification happens or certain event happens and then a closure or callback is called automatically for you, which you registered before, which is really nice. But when we started, there was none, it was not that type of event handler, so we couldn't use it. We didn't get around reworking our code yet. So now let's have a quick look at the practical itself. After starting practical, you see a quick loading spinner, it updates the state, executes the sync, obviously the loading time depends on when it was last synced, but I did it quite recently, so it was really fast this time. The UI is pretty similar to previous versions of practical, except that it got a phase lift and it was updated to the new style guides for now. So let's quickly look at what we see here. So on the left, we have a list of rooms in different categories and we even can move them between categories like what are we going to move, I don't know, GK room for example, we can left click Marker's favorite, it disappears, now it's up there. And if we want, we can open it and then you choose the room history, it even loads later events. This is already the case. What else can we do? We can move it back to favorites, unfavorited to normal rooms, then we can explore new rooms, join them, for example, let's join the practical room. Oh, actually, we are already in the practical room, so we can join, but we can immediately view it from here, which is pretty nice. It takes some time to load. Okay, here we go. Some nice hearts, me reacting to it, that's a really nice feature implemented by Kevin quite easily and I'm excited about it because practically not half it, although the previous version I can not because that's not set with this account, but I can add one like a heart and remove it again. And everything, yeah, it feels pretty solid from this, but we're still missing features, we even can send messages. Live demo, my name is Jesse Messi in this case. So what else do we have? We have basic room details, this account cannot modify them, modify properties of the room because it's just a test account. As well we have a list of members with admin rights, pretty much similar to the previous practical version. We can invite people, I know, let's invite Julian, myself. Oh, actually, I forgot. Users that are already in the room, I exclude from the search. So I can't invite myself. Let's include fractal test count. Oh, Johan fake, we can invite him. Let's invite him. Okay, here we go. Jeff fake was invited to this room, what else can we do? Something really interesting and fancy, I can implement during his GSOC as a multi-account support. Like, I could add a second account here, but let's not do that right now. And use it with multiple accounts. Then we have device list. Maybe you noticed there is a small shield, which means that fractal access currently verified. Here below we have a session with point not verified. We can remove it. And obviously it asks for authentication. Hello, what else, what else? We can join new rooms. Create new rooms, I meant to say. Inspect the source. Oh, nice, we even have gift support. This was not planned, actually. We can see a dancing bear. So if you want to help, then please do come talk to us on matrix or have a look at our repository and we will happily help you and direct you to something you can do. And yeah, thanks for your attention. And again, I'm Julian. And if you have some questions, then feel free to ask them. All right, thank you both for your talks. I've got Julian and Paulie are here with me for live Q&A. And we're going to start with the first question for Julian on fractal. So NLNet was provided funding for fractal and that was super important to get the core of fractal next to be written. But did you get an interest from potential paying customers in fractal next and are you open for potential sponsored work on it? So yeah, actually, we did not yet get any interest from any other site and NLNet to sponsor our work, but maybe in future, I would definitely be open to do some work on fractal next with the company sponsoring it. At this point, I want to also mention that the NLNet grant is pretty much over or it's actually already over. And now I'm working on it on my free time. So it also would be really nice to have a company sponsor to work to start finish fractal next and get the release out there. Right, so you've heard it guys. If you want to see some new cool stuff in fractal and you want to sponsor it, just contact Julian. One for Paulie now. So is the matrix rest SDK entirely spec compliance or if not, what are the outstanding missing features? So spec compliance is generally handled by Roma, who handles all the passing and from HTTP responses into requests, mainly things missing things are knocking spaces, a bunch of smaller things in the capital area. But yeah, we are targeting it all of those. It's always cross to changes in Roma and then the rest of the system. Right. And do you have a rough estimation on how we should expect the rest of the state to be fully spec compliant? We're always playing catch up because up until now it was so. So one main throw by me, we're now getting more people to work on the rest SDK. So maybe this year we will catch up finally with all this back work. All right. Excited to, looking forward to see that improve. Back to Julian now. What is, can you rough summary of what's missing in practical makes before it can be officially released? So that's actually quite the question because it's still missing a lot. But on the other hand, we already implemented a lot of stuff we did not have in the previous version of Rackfall. So I think a major point is still the account settings. Like you can't configure your account. You can change your username or anything like that. Account registration probably also we need to add SSO. So single sign on which somebody is working on a volunteer. What else? Like if in a room history, some keys are missing to decrypt the message. We don't request them or we don't show it in the UI. And it will just show forever that the message couldn't be decrypted. And in general, a little bit of polish and probably I'm forgetting a lot. But from the code base, it's pretty easy to add stuff. But we still need to do it. Yeah, definitely. Looking forward to seeing a fractal next in the coming future. Another question for Paul, yeah. Is butter the Mac? I'm pretty sure I'm pronouncing it wrong. Using other crates for crypto or does it re-implement absolutely everything? Yeah, of course we are using other crates. I think I mentioned that in the talk that we're using mostly crates from the Rust crypto project. Those are Rust implementations. So crypto permitives, we're using the advanced and triple standard rate there. And the ED25519 and the X25519, all the implementations I used for the double register. Makes sense. Now this question, I'm not entirely sure which one of you guys is best place to answer that, maybe Julian. What features were missing from the Rust SDK that were required for fractal? So first of all, what was missing was a semi-polished high level API to interact with rooms, which was not there a year ago, and now it got way better. Also we were missing a good way to get new room events, which now we have the streaming API, which was added by Blathe, I think, which we actually don't use in fractal yet, but it's very nice. Most of the time I actually spend on the message store so that the entire room timeline is stored in the SDK, but I also wasted a lot of time on that because I tried to store all the events, but Matrix does not actually allow that because it does not give you diffs of the sync. So if the timeline changes or events get pushed into the timeline because of federation or something like that, we don't really know how that works or how we get those updates, so we can't have a precision local store. But now I have a good plan to do it, not completely persistent, but more like a cache, and that should be published soonish in a week or so. Oh, right. So it will be added. All right, now for maybe the final question, given – so that's one for Paul, yeah. Given you're the maintainer of both the Matrix, Resist, SDK and Neo, are there chances that we can see Matrix, Resist, SDK powered Neo in the coming months? Hard to tell. I really want to do this, but there's really a lack of time, and not many people are that interested in the Python thing, just various other findings. The thing obviously won't be interested in anything else currently, except that there must be a crypto-grade, we are making a switch client that won't touch anything that's Python-land, it won't generate any findings like the crypto-grade generates, various languages that will use its own. So it's really a wanted feature, but nothing really touches Python in that case. At least I am aware of it. Right, yeah. So yeah, we can talk to – see what happens, when it happens, but yeah, definitely be aware of the process, if it can happen, just now, and now, probably definitely the last question this time, what kind of bindings does the Resist SDK expose for non-resusers? All right, we just lost all. We lost them all. Yeah, we lost all Resist SDK experts, and in a timely way, because the Q&A is reaching the end, so thank you both for your great thoughts, and have a great rest of the conference. Thanks, Yutu. Thank you for having me.
 Hello, Faslem.  Welcome to my quick talk about the Matrix Rust SDK.  The Matrix Rust SDK is the Matrix Client Library.  This talk will be in two parts.  I'm first going to talk about the Rust SDK, how the SDK came to be, what's designed, a  case study, how it's used in Android land.  And after that, we're switching to Julian's father, who will talk about Fructal, a GTK  Matrix Client written in Rust, which heavily uses the Rust SDK.  Before I start, a little bit about me.  I'm Damir.  I'm rural Croatia, and since there's nothing to do in rural Croatia, I don't have some  alcoholic or thanks to the advances in technology and work.  So I thought to myself, why not do both?  Before we jump directly to the Rust SDK, let's first talk about history a bit.  How did we end up with the Rust SDK after all?  Thanks for starting when I was shopping around for a bit to chat with close friends in a  secure manner.  I was already using BeatShot for ISC and XMPP, and I didn't want to change my work.  The classic XKCD, don't change our man's workflow, if you remember that one.  The one could use off the record messaging with XMPP, but that turned out to be a bit  complex for non-tech friends.  For those who don't know what BeatShot is, it's a text user interface ISC Client written  in C and some 18 years old.  It's written in this modular way that cross essentially a chat-specific text user interface  framework, and everything else is a plugin.  The ISC support is a plugin as well.  You can expand the client to support any type of protocol.  Now there are a bunch of assumptions that the text user interface has that are ISC-specific,  so you might sooner or later run into trouble with some different protocols.  Here's what we are dealing with.  It's a single-fet it program.  It's event loop-based and call-back-based.  Extensions can be written in a bunch of languages.  It can't spawn threads unless you're writing your extension in C.  It can open sockets for you.  It's important it's opening sockets in a separate thread to not block the UI thread.  I have to mention that there already was support from Matrix implemented as a new extension,  though the end-to-end encryption support became stale.  It didn't support the latest version and has had various other problems.  I started writing a new extension in Python, and I was a bit annoyed that I couldn't use  at the time existing Python SDK.  This nudged me towards making sure that whatever I write for Richard Matrix should be reusable  for other projects and plans.  Therefore, other featured Matrix came new, a modular Python client lab with a no-year  API, a very push-pull HTTP request in and out of state machine and an async API on top  of that.  Sadly, there are still problems with this, at least for the Richard use case.  End-to-end encryption support in Matrix requires sending out multiple requests in order.  That quickly leads to callback hell if you're not using async arrays like we can't use  async arrays and reach it.  Another problem is that there's no multiple threading, and expensive cryptographic operations  might block the UI.  If you're sending an encrypted message, the encryption keys need to be sent out to a bunch  of devices.  You need to encrypt for each device separately, and this might take some time, depending on  the size of the room.  And in that meantime, the UI will block.  So relatively early on, I started to experiment with Rust bindings for the Richard plugin  API and a accompanying Rust Matrix client library.  This leads us now to the Rust SDK.  It also has modular structure.  These names are all trade names except the state cell.  This one should become in the future a separate trade as well.  You could pick any of these trades and integrate it into an existing client library, possibly  in another language, or you could pick the whole Matrix SDK trade and blend it into  another language.  I should note that we are using Vroomer, so input validation layer.  It passes all the responses into high level types, so we don't need to deal with JSON  directly.  So the Matrix SDK trade here should be the most interesting one for library developers.  We are developing a Matrix client library, which doesn't yet support end-to-end encryption.  Check out the crypto trade.  As a simple push-call API, you pull HTTP requests out of the lib and push HTTP responses into  the lib.  You don't need to understand the complicated logic of how these responses need to be handled  or passed or which requests need to be sent out when and how to verify signatures, signed  objects, all of that is handled transparently in the crypto trade.  As for the Matrix SDK trade, if you remember the code sample from Neo, we mostly retained  a relatively simple Python API.  Most things should be quite straightforward and end-to-end encryption support should be  completely transparent.  Messages get encrypted automatically and possible decrypted as well.  Now this is really similar to Neo.  We are using more types obviously because we are in Rustland, but all the flow is kept  really similar.  If you are familiar with Neo, this should be an easy upgrade there.  But also if you are not familiar with Neo, it should be relatively simple to write things.  Of course, if you are interested in writing bots, this might be a little bit low-level  for you.  If you are interested in a higher-level bot API on top of the Rust SDK trade, it would  be the way to go.  A bit more about the data flow in these trades.  I already mentioned briefly that we have this proofable design.  At the top here lies the home server, which has data.  The Matrix SDK trade does not sync HTTP requests, which as for data, this request will contain  an initial state.  Some sequent calls will contain only the delta changes from the previous state.  We push the response into the Matrix SDK base trade and into the Matrix SDK crypto trade.  Those two trades contain state machines that consume the HTTP responses, update the internal  state and finally generate some outgoing requests.  Those requests are then important to keep the encryption going or to keep your state  correct.  You need to track other people devices periodically, upload new cryptographic keys to the server.  Basically everything that needs to be sent to the server to have working end-to-end encryption  support will be pushed out of the state machine as a group request that needs to be sent to  the server.  This does mean that all our networking IOS is having in the Matrix SDK trade, while base  and crypto trades behave mostly as pure state machines.  Some persistence is there, like data we are possessing encryption keys and to a state  and so IOS is not pulled out of the trades but is abstracted away as a trade.  Whereas the Matrix SDK used, other dimensions, fractal, this is a high-level, probably flagship  plan that is using the SDK.  It supports encryption, it supports QR code verification, emoji verification.  Every advanced feature should be supported by fractal.  Richard Matrix is my rewrite of Richard Matrix.  It has become a bit stale due to lack of time but eventually it should replace Richard Matrix.  We have the Matrix SDK app service crate.  This one is built on top of the SDK and it's used to build ridges and privilege bots for  various use cases.  Some things only use the crypto crate.  Element Android, for example, uses the Matrix SDK crypto crate and the Matrix Bust SDK  as well uses the crypto crate.  Element Android is written in Kotlin and Matrix Spot SDK is written in JavaScript and there  are bindings from Rust to Kotlin and JavaScript.  Let's take a closer look into how the Rust SDK is used in Element Android.  Do note that this isn't yet in production but in a late beta phase.  It only uses the crypto crate and we will take a look at how this is integrated.  Firstly, we are using Mozilla's UniFFI crate to mostly generate Kotlin bindings.  This is a bit different if you are familiar with Buzz and Buncheon and PiO3.  It's a bit different from those crates.  You have a separate file where you define your interface and then it generates Kotlin  bindings or JavaScript bindings or Python and Ruby as well bindings from the separate  definition.  Our mind structure here is called the All Machine.  This is the state machine that handles the cryptography and let's take a look at the  code snippet.  This is the first method that we need to bind.  This is skipping the All Machine initialization but creating all machine is quite simple.  We just pass a user ID and device ID and if you remember our data flow graph, this method  is the push method.  It pushes changes we receive from the server into the state machine.  You don't need to push the whole sync response into the state machine.  You just push the smaller parts into it.  Those are all directly tied to the crypto support.  The two device messages will contain room keys, the device changes are important to  track which devices a user has and the key counts are important so we know which keys  we need to replenish from the server.  The two device events here will then be decrypted and returned by the All Machine and you can  then consume those events separately.  Next up is a pull method.  This pulls all the HTTP requests out that are either here because we first initialize  the All Machine or the sync changes produced.  There are a bunch of request types we will get from this.  I have not listed them all.  It's best to send them all out in parallel and you need to notify the All Machine after  each request is sent out.  The response needs to be passed back to the All Machine so it knows you sent out this  request.  One of the requests fails to be sent out which can happen especially on mobile connections.  SDK will keep answering you with the request until you tell that this has been sent out.  This is a common pitfall with various clients.  People tend to forget to try sending out a request and that request might have sent the  encryption keys to other people and suddenly people can encrypt the messages.  As a backup plan, if you fail to retrieve a request sending, the All Machine will have  your back and let you know you need to send this out.  This isn't done for all request types.  Some of them are of memorial after all but the most important ones for example the Groom  keys, the decryption keys will be kept in memory until you send them out.  That's it.  Now you can decrypt the messages.  This doesn't happen automatically.  We have a separate method for this because we don't know what your user is looking at  and you want to prioritize decrypting the messages that your user is looking at.  So you can do that quite easily.  Now for the sending side, we need to do two things.  We need to establish one-to-one secure channels between each device we're going to talk to  and we need to send each of these devices a room key encrypted using the secure channel.  So you first ask the All Machine, hey, who doesn't have this secure session?  And it returns a request and after you pass the response to the All Machine, mostly everyone  will have the secure channel established.  After that, we call the share room key method and this will encrypt the room key for this  given room and for those members and this will generate either one request or a bunch  of requests depending how large the room is and you can send those out in parallel as  well.  And if you fail to send some of these, the All Machine will tell you, hey, you need to  send this out the next time you call a share room key method.  And that's it.  Now you can encrypt messages as well.  Well, almost.  I skipped one method where you need to tell the SDK which users are part of encrypted rooms  so it can track devices users have.  But even so, I think this is quite a simple API compared to what you need to do if you  implement this from scratch.  Let's take a quick look at the performance of Element Android using the Matrix SDK Crypto  trade, if until now nobody has been convinced to use the Rust SDK Crypto trade, perhaps  this will convince them.  This has been already explained to reiterate and rate the client to send an encrypted message  if they just need to share what we call a room key with every device in the room.  To do this, a room key needs to be encrypted for each device separately.  And this graph shows how long clients take to share this room key.  Now this isn't an apples to apples comparison since we use different devices, but the slowest  device should definitely be the Android mobile phone.  And this is the mobile phone, Android mobile phone is the one that's just using the Rust  SDK.  So here the corroded and rates on the graph are Element Android converted to use the Crypto  trade of the Rust SDK.  So we managed to lower the time required for minutes for seconds.  Now this isn't all because it's Rust or because we're using multi-threading.  There are certainly problems in Element Android, but I don't think anybody uses multi-threading  to encrypt room keys.  And therefore I don't think anybody comes close to the performance of the Rust SDK.  And sharp-eyed people might notice that there are two versions of the corroded Android code  base.  One is neatly called Wothers Emmets.  And we'll get to that next.  What does this mean?  So a bit of a discretion here and what does Wothers Emmets actually?  Wothers Emmets is the craving name of amphibians.  Why amphibians?  Well, amphibians have some amazing self-healing properties.  Like a gecko, you catch the tail of a gecko, the gecko will discard the tail and it will  regrow.  And because the double-red should protocol, which is used in Matrix, has amazing self-healing  properties as well.  Implementations were usually named as some amphibian.  The original implementation for TechSecure, the predecessor of Signal, was called Exolotl.  The Matrix version was called all or is called all.  And Wothers Emmets is a pure Rust implementation of the Matrix variant of the double-red for  Tupel.  So why would you ever rewrite your cryptographic library?  Why would you ever rewrite anything in Rust?  Yes, LeBolm was stable and widely used.  But let's take a look at the API of LeBolm and see why we might not like it that much.  This snippet creates an ALMA account.  This is basically the identity of your device.  Each one of your devices will have its own ALMA account.  So this involves a couple of steps.  Let's ask LeBolm how many bytes we need to allocate.  Ask LeBolm how many bytes of randomness we need to generate, allocate the memory and generate  the randomness, and then call the initialization method.  So this is a five-step process.  And basically, anything you do will have the same layout here.  And this shows us a couple of problems with LeBolm.  It has a vitri-type API.  You pass in a bunch of opaque buffers.  If you reuse your buffers, you might get a slightly incorrect result.  That's hard to notice.  So you have your input buffer, you have your randomness buffer, you have your output buffer,  and all of those could be interchanged without anybody noticing.  This has actually happened to us where we got a slightly incorrect base 64 encoded message.  So the input buffer was reused as the output buffer.  So the base 64 encoder was encoding an already encoded thing.  So it was slightly incorrect and hard to reproduce a bug in another library.  So all of that happened because buffers are all opaque and you don't know what you're  passing in.  And the API is quite complicated.  You need to make multiple calls to generate randomness, allocate things, and every language  binding of the library will need to again allocate things and generate randomness separately.  And the next one here is scratch-dangerous.  As you find allocation size might return negative value.  Every time you want to do something, you ask the library how many bytes you need to allocate.  And there are a bunch of such methods.  Some of them might return on error, some of them can't.  If you forget to handle this error, which will be returned as a negative value, you  might try to allocate all of the bytes because the negative value will be truncated to a  very large value.  And you possibly just edit denial of service into vulnerability into your client.  Furthermore, the underlying libraries that are used for the cryptographic primitives  crypt255.9, ed255.9, and encryption standard run that create either.  So these are just some problems of LeBone that it shows already that people need to  work a lot of LeBone to get it to a modern state.  So let's take a look at what does MADS API that creates an account.  These snippets are three different languages and all of them create an account in the same  way.  You just call the constructor and everything gets allocated and randomness gets generated.  This isn't really a criticism of C or LeBone.  At the time, it was probably the best idea to leave their allocations to various platforms.  But nowadays, the tooling will take care to use the correct implementation on the VASM  and Python and Rust.  And this looks much simpler, right?  Bindings become much easier and can have the same high-level API as the Rust version.  The bindings also don't allocate or generate randomness anymore.  So here's what the MADS offers.  MADS high-level API.  Thanks to Rust, it's easily bound into different languages.  Using VASM, VIN, and BioFree, you name your language binding straight.  We are using modern implementations of cryptographic primitives from Rust, crypto, and from the  Dalek traits, so X25519 Dalek and DD25519 Dalek.  And all of those have been audited.  We are also going to audit for the Zemmats.  And of course, we provide an upgrade app from LeBone.  So LeBone has a way to persist all the objected traits.  And for the Zemmats, you can read the data format LeBone uses.  We are still not sure what data format for the Zemmats will use.  Maybe a server-based or maybe it will use the binary data format LeBone uses.  And the API is quite similar to what different on-bindings were exposing, so it shouldn't  take too long to switch to all those Zemmats.  And this concludes the brief overview into the Rust SDK and what else around it is happening  in.  The rest of the talk will take a deeper look into Fractal, how it works, and how it utilizes  the Rust SDK.  And this part will be explained by Julian Faber as mentioned, and off to Julian.  Bye.  Hi, all.  I'm Julian.  I'm going to talk about Fractal today and the journey of writing it.  So Fractal is a native matrix client built for the GNOME desktop.  So it follows the design principle laid out by the GNOME design team.  It works closely with them.  And therefore, it has a huge focus on usability as well as on user experience.  So before we dive into the project, let's have a quick look at the project's history.  So it was startled all the way back in 2017 by Daniel Brasio Moreno, who wanted to build  a GTK client for Rust.  And therefore, he forked Rumor GTK, which now is called Festa I Am, written by Jonas  Blathe, even though it did not see much development since the fork, actually, or in the last couple  of years.  But Fractal, on the other hand, got huge attention over the last years.  We even participated multiple times in GSOC and Outreachy.  And I have been one of the GSOC students and later than also as Manchur, which was quite  fun.  And yeah, now five years.  So it has been a long time.  But now what we are going to do is Fractal complete.  I can answer that quickly.  No, it's not.  We're actually doing something interesting.  Back in February, so a year ago, obviously a year ago, we decided to do a rewrite.  A rewrite?  Yes, a rewrite.  But this time with N2N encryption support.  Fractal, even though we wanted it from the very beginning, didn't ever have encryption  support because Fractal started out as a pre-time project.  And it has been most of the time run by or developed by volunteers.  So it was really hard to get something like encryption support built in your pre-time,  actually.  So we now got around.  We keep moving it.  And yeah, but this time it's going to be different.  We did it from the beginning and now I'm going to tell you a couple of reasons why we decided  to do the rewrite.  So first of all, probably new.  Already it's made with Rust SDK.  It's pretty new.  It's another version.  But it's very usable and we also spent a lot of time to contribute it and provide feedback  to it so we could use it.  And we managed to build a client on top of it, which is not finished yet, but I will  show it later when we start.  The next reason is GTK4 got released a year ago, which is the UI toolkit we use to build  Fractal.  Initially we used GTK3 because that was the most recent version we could use.  And now we have GTK4, which comes with a lot of improvements, as well as the bindings to  Rust because we are talking about Rust Word.  GTK is written in C. It got huge improvement, the so-called bindings, language bindings,  which connect GTK4 into the Rust Word.  And all the people, a shout out to them, they did an awesome job improving it and working  on it the last few years, probably for quite some time.  And now it's changed completely.  Therefore our code base was pretty much outdated because we could use all the new fancy features  of GTK4 and new bindings for it.  Last but not least, LibAdvaita was released.  Quite recently, the stable release was released quite recently, like two months ago or three  months ago.  And that's library, the successor of LibHandy.  Maybe you heard of it already.  It's library to build responsive GTK apps, adaptive GTK apps.  So Fractal will run also on small screen sizes, for example, mobile phones.  And actually LibAdvaita is much more than just a library to build adaptive apps, but  also a building block for every modern application.  So all of that sounds like it would reason, right?  And yes, you are right.  It is.  So after first trying to integrate the major Rust SDK into mostly by Hando, who also was  a GSOC student, into the old Fractal code base, we realized quickly that it's not going  to work because so much things changed.  So then in February, as I am a couple of times already said, we launched the initiative  Fractal Next.  So now it's all about rewriting the Fractal in native metrics time.  So at this point, I need to mention that this effort would not be possible in my free time  or that matter for anybody, probably, to build a project in that time span with that many  hours working on it if it was not financed by somebody.  We are lucky.  I applied last year for funding from an internet and we want to grant as part of the private  and trust enhancing technologies fund to implement and to enter encryption in Fractal.  So the initial plan was always to use the matrix SDK and build on top of that, but the  idea was not to rewrite Fractal initially.  But after a couple of months, we pretty soon realized that it does not make sense to spend  more time on the current code base or back then the old code base.  Therefore, we asked for an extension and they were happily happy to support us also by,  but they rewrite the accomplished out.  And an internet gets most of the money from the EU Commission pretty much directly, so  it's sponsored by the EU Commission, you can say.  Additionally, I want to thank Kevin, Tobias, Kajar, and Alejandro for having a lot during  the last few months, Kajar and Alejandro mostly during the summer months, Kaj the other time,  Tobias with design support during the entire project and it's still not finished, just  to mention that again.  So now let's get a little bit more technical and see what's happening inside Fractal.  So as you probably heard in the previous part of this talk, the Rust SDK is multi-freted  and uses as an asynchronous framework Tokyo, not early, but mainly that one.  On the other side, Fractal uses GTK to draw widgets and UI, which requires it to be single  fretted.  Obviously, this does not mean that Fractal can use multiple frets and we also use multiple  frets.  So the idea we developed is that we have a main frett called UI frett that handles all  the GTK things.  And on the other side, we have Tokyo running tasks needed for the Rust SDK, like making  requests and everything the Rust SDK does for us.  And to communicate between those two, we use different approaches, or actually two different  approaches.  Mostly, we use channels and features, which are if you're familiar with Rust or in general  any modern language, you probably know them already, they allow you, the ways or channels  are ways to communicate between frets or tasks, which we do in the case of Fractal.  Furthermore, we need to wrap all the Rust trucks and matrix events, matrix trucks, the matrix  Rust trucks into the object so that we can have a data model which can then be bound to  the UI, which we have already.  Additionally, the matrix Rust SDK provides mostly an asynchronous API, which does not  mean that data is not immediately available, but we can rely on that because it may be  on a remote server and the matrix Rust SDK still needs to request that information.  Therefore, we do some local caching inside the G object, which is pretty much an object  holding data, which then on the other hand provides us with a synchronous access to the  data so that the UI feels snappy and reactive.  So, matrix, one of the most important building blocks as the sync request, and also in Rust  SDK, you need to prepare the sync to synchronize the state between the home server and your  local machine.  Most of that is handled actually by the Rust SDK, which is pretty nice, but there are  different ways to obtain the sync response from the SDK.  One is to manually make the sync request and do the SDK handle anything, which obviously  you probably don't want because then you have to handle everything yourself, like encryption,  key sharing and that stuff.  Then there is another version where you get a stream, so it's a stream of sync responses  and you can read from the stream and the matrix SDK will push new responses it requests from  the home server to the stream.  Maybe we are using that version and move it to a channel to be able to update our G objects,  so our local cache of the state of your client or for example, a root stage or something  like that.  The third option, which is quite new and we would like to switch to that sooner or later,  but we did not get around is that you can register event handlers, so you can say something  like or write something like I want to get notified when a certain notification happens  or certain event happens and then a closure or callback is called automatically for you,  which you registered before, which is really nice.  But when we started, there was none, it was not that type of event handler, so we couldn't  use it.  We didn't get around reworking our code yet.  So now let's have a quick look at the practical itself.  After starting practical, you see a quick loading spinner, it updates the state, executes the  sync, obviously the loading time depends on when it was last synced, but I did it quite  recently, so it was really fast this time.  The UI is pretty similar to previous versions of practical, except that it got a phase lift  and it was updated to the new style guides for now.  So let's quickly look at what we see here.  So on the left, we have a list of rooms in different categories and we even can move  them between categories like what are we going to move, I don't know, GK room for example,  we can left click Marker's favorite, it disappears, now it's up there.  And if we want, we can open it and then you choose the room history, it even loads later  events.  This is already the case.  What else can we do?  We can move it back to favorites, unfavorited to normal rooms, then we can explore new rooms,  join them, for example, let's join the practical room.  Oh, actually, we are already in the practical room, so we can join, but we can immediately  view it from here, which is pretty nice.  It takes some time to load.  Okay, here we go.  Some nice hearts, me reacting to it, that's a really nice feature implemented by Kevin  quite easily and I'm excited about it because practically not half it, although the previous  version I can not because that's not set with this account, but I can add one like a heart  and remove it again.  And everything, yeah, it feels pretty solid from this, but we're still missing features,  we even can send messages.  Live demo, my name is Jesse Messi in this case.  So what else do we have?  We have basic room details, this account cannot modify them, modify properties of the room  because it's just a test account.  As well we have a list of members with admin rights, pretty much similar to the previous  practical version.  We can invite people, I know, let's invite Julian, myself.  Oh, actually, I forgot.  Users that are already in the room, I exclude from the search.  So I can't invite myself.  Let's include fractal test count.  Oh, Johan fake, we can invite him.  Let's invite him.  Okay, here we go.  Jeff fake was invited to this room, what else can we do?  Something really interesting and fancy, I can implement during his GSOC as a multi-account  support.  Like, I could add a second account here, but let's not do that right now.  And use it with multiple accounts.  Then we have device list.  Maybe you noticed there is a small shield, which means that fractal access currently  verified.  Here below we have a session with point not verified.  We can remove it.  And obviously it asks for authentication.  Hello, what else, what else?  We can join new rooms.  Create new rooms, I meant to say.  Inspect the source.  Oh, nice, we even have gift support.  This was not planned, actually.  We can see a dancing bear.  So if you want to help, then please do come talk to us on matrix or have a look at our  repository and we will happily help you and direct you to something you can do.  And yeah, thanks for your attention.  And again, I'm Julian.  And if you have some questions, then feel free to ask them.  All right, thank you both for your talks.  I've got Julian and Paulie are here with me for live Q&A.  And we're going to start with the first question for Julian on fractal.  So NLNet was provided funding for fractal and that was super important to get the core  of fractal next to be written.  But did you get an interest from potential paying customers in fractal next and are you  open for potential sponsored work on it?  So yeah, actually, we did not yet get any interest from any other site and NLNet to  sponsor our work, but maybe in future, I would definitely be open to do some work on fractal  next with the company sponsoring it.  At this point, I want to also mention that the NLNet grant is pretty much over or it's  actually already over.  And now I'm working on it on my free time.  So it also would be really nice to have a company sponsor to work to start finish fractal  next and get the release out there.  Right, so you've heard it guys.  If you want to see some new cool stuff in fractal and you want to sponsor it, just contact  Julian.  One for Paulie now.  So is the matrix rest SDK entirely spec compliance or if not, what are the outstanding missing  features?  So spec compliance is generally handled by Roma, who handles all the passing and from  HTTP responses into requests, mainly things missing things are knocking spaces, a bunch  of smaller things in the capital area.  But yeah, we are targeting it all of those.  It's always cross to changes in Roma and then the rest of the system.  Right.  And do you have a rough estimation on how we should expect the rest of the state to  be fully spec compliant?  We're always playing catch up because up until now it was so.  So one main throw by me, we're now getting more people to work on the rest SDK.  So maybe this year we will catch up finally with all this back work.  All right.  Excited to, looking forward to see that improve.  Back to Julian now.  What is, can you rough summary of what's missing in practical makes before it can be officially  released?  So that's actually quite the question because it's still missing a lot.  But on the other hand, we already implemented a lot of stuff we did not have in the previous  version of Rackfall.  So I think a major point is still the account settings.  Like you can't configure your account.  You can change your username or anything like that.  Account registration probably also we need to add SSO.  So single sign on which somebody is working on a volunteer.  What else?  Like if in a room history, some keys are missing to decrypt the message.  We don't request them or we don't show it in the UI.  And it will just show forever that the message couldn't be decrypted.  And in general, a little bit of polish and probably I'm forgetting a lot.  But from the code base, it's pretty easy to add stuff.  But we still need to do it.  Yeah, definitely.  Looking forward to seeing a fractal next in the coming future.  Another question for Paul, yeah.  Is butter the Mac?  I'm pretty sure I'm pronouncing it wrong.  Using other crates for crypto or does it re-implement absolutely everything?  Yeah, of course we are using other crates.  I think I mentioned that in the talk that we're using mostly crates from the Rust crypto  project.  Those are Rust implementations.  So crypto permitives, we're using the advanced and triple standard rate there.  And the ED25519 and the X25519, all the implementations I used for the double register.  Makes sense.  Now this question, I'm not entirely sure which one of you guys is best place to answer that,  maybe Julian.  What features were missing from the Rust SDK that were required for fractal?  So first of all, what was missing was a semi-polished high level API to interact with rooms, which  was not there a year ago, and now it got way better.  Also we were missing a good way to get new room events, which now we have the streaming  API, which was added by Blathe, I think, which we actually don't use in fractal yet, but  it's very nice.  Most of the time I actually spend on the message store so that the entire room timeline is  stored in the SDK, but I also wasted a lot of time on that because I tried to store all  the events, but Matrix does not actually allow that because it does not give you diffs of  the sync.  So if the timeline changes or events get pushed into the timeline because of federation or  something like that, we don't really know how that works or how we get those updates,  so we can't have a precision local store.  But now I have a good plan to do it, not completely persistent, but more like a cache, and that  should be published soonish in a week or so.  Oh, right.  So it will be added.  All right, now for maybe the final question, given – so that's one for Paul, yeah.  Given you're the maintainer of both the Matrix, Resist, SDK and Neo, are there chances that  we can see Matrix, Resist, SDK powered Neo in the coming months?  Hard to tell.  I really want to do this, but there's really a lack of time, and not many people are that  interested in the Python thing, just various other findings.  The thing obviously won't be interested in anything else currently, except that there  must be a crypto-grade, we are making a switch client that won't touch anything that's  Python-land, it won't generate any findings like the crypto-grade generates, various languages  that will use its own.  So it's really a wanted feature, but nothing really touches Python in that case.  At least I am aware of it.  Right, yeah.  So yeah, we can talk to – see what happens, when it happens, but yeah, definitely be  aware of the process, if it can happen, just now, and now, probably definitely the last  question this time, what kind of bindings does the Resist SDK expose for non-resusers?  All right, we just lost all.  We lost them all.  Yeah, we lost all Resist SDK experts, and in a timely way, because the Q&A is reaching  the end, so thank you both for your great thoughts, and have a great rest of the conference.  Thanks, Yutu.  Thank you for having me.
matrix_rust_sdk_0_155.png matrix_rust_sdk_156_935.png matrix_rust_sdk_936_1367.png matrix_rust_sdk_1368_1607.png matrix_rust_sdk_1608_3203.png matrix_rust_sdk_3204_4775.png matrix_rust_sdk_4776_6791.png matrix_rust_sdk_6792_8819.png matrix_rust_sdk_8820_10319.png matrix_rust_sdk_10320_12911.png matrix_rust_sdk_12912_15203.png matrix_rust_sdk_15204_16859.png matrix_rust_sdk_16860_18851.png matrix_rust_sdk_18852_22151.png matrix_rust_sdk_22152_24875.png matrix_rust_sdk_24876_27587.png matrix_rust_sdk_27588_27731.png matrix_rust_sdk_27732_29111.png matrix_rust_sdk_29112_30767.png matrix_rust_sdk_30768_34799.png matrix_rust_sdk_34800_36287.png matrix_rust_sdk_36288_38243.png matrix_rust_sdk_38244_38807.png matrix_rust_sdk_38808_39191.png matrix_rust_sdk_39192_45155.png matrix_rust_sdk_45156_46955.png matrix_rust_sdk_46956_49379.png matrix_rust_sdk_49380_53075.png matrix_rust_sdk_53076_56231.png matrix_rust_sdk_56232_58295.png matrix_rust_sdk_58296_62507.png matrix_rust_sdk_62508_62771.png matrix_rust_sdk_62772_66155.png matrix_rust_sdk_66156_66911.png matrix_rust_sdk_66912_68039.png matrix_rust_sdk_68040_68147.png matrix_rust_sdk_68148_68963.png matrix_rust_sdk_68964_70511.png matrix_rust_sdk_70512_70607.png matrix_rust_sdk_70608_71051.png matrix_rust_sdk_71052_71567.png matrix_rust_sdk_71568_72995.png matrix_rust_sdk_72996_73043.png matrix_rust_sdk_73044_73235.png matrix_rust_sdk_73236_73427.png matrix_rust_sdk_73428_73487.png matrix_rust_sdk_73488_74675.png matrix_rust_sdk_74676_75179.png matrix_rust_sdk_75180_75359.png matrix_rust_sdk_75360_75407.png matrix_rust_sdk_75408_75515.png matrix_rust_sdk_75516_76439.png matrix_rust_sdk_76440_76715.png matrix_rust_sdk_76716_77219.png matrix_rust_sdk_77220_77555.png matrix_rust_sdk_77556_77711.png matrix_rust_sdk_77712_78527.png matrix_rust_sdk_78528_80063.png matrix_rust_sdk_80064_80231.png matrix_rust_sdk_80232_80351.png matrix_rust_sdk_80352_80759.png matrix_rust_sdk_80760_81479.png matrix_rust_sdk_81480_81911.png matrix_rust_sdk_81912_83807.png matrix_rust_sdk_83808_84611.png matrix_rust_sdk_84612_84719.png matrix_rust_sdk_84720_84899.png matrix_rust_sdk_84900_85439.png matrix_rust_sdk_85440_85943.png matrix_rust_sdk_85944_86003.png matrix_rust_sdk_86004_86603.png matrix_rust_sdk_86604_87107.png matrix_rust_sdk_87108_87671.png matrix_rust_sdk_87672_87803.png matrix_rust_sdk_87804_88115.png matrix_rust_sdk_88116_88163.png matrix_rust_sdk_88164_88295.png matrix_rust_sdk_88296_88403.png matrix_rust_sdk_88404_88523.png