Andy Piper

2024-12-24 17:20:22

Recently, I delivered a keynote speech at RubyWorld 2024 in Shimane, Japan.
Andy Piper on stage at RubyWorld 2024
I was there on behalf of the Mastodon project – Mastodon being an example of Ruby and Rails in production at some degree of scale. My talk was much broader than the scope of just our segment of the social web, and was more about the state of the internet today, and how we must all make choices that enable the better web that we say that we want – reflected in the title: The Social Web: Decentralization and Building a Better Internet.
Andy Piper standing at the RubyWorld podium
It was truly an honour to be invited to deliver this presentation, and a delight to be back in Japan, five years after my previous visit. Matsue is a lovely, quiet, friendly city. I really enjoyed the trip.
Screenshot of the English livestream of the keynote on YouTube
There were a couple of last-minute changes and realtime rewrites! Among other things, as I woke up on the morning that I was due to deliver the talk, I read the news that Threads had just enabled the first element of two-way federation¹, and I had to quickly update my script. An off-the-cuff update would not usually have been an issue, but I was also working with an excellent team of simultaneous interpreters who had seen the original planned version of the text, so I was grateful for their support as I made the changes.

I’m including the full spoken transcript of my talk below, with slides available here; the video version will be available in future, but I believe only with a Japanese translation overdub (a technical issue prevented a complete version in English from being saved).

I have a handful of the special Japanese stickers that I had made for the RubyWorld event left over, so come find me at FOSDEM 2025 in February if you’d like one!

---

In related but separate news: the Mastodon team just published our annual report for progress during 2023. I’ve seen a number of ironic comments in response to the report being announced on Mastodon, related to the fact that it is almost a year “out of date”, but if you take the time to read before posting something funny about the date, you’ll find that we readily admit that it was delayed, primarily due to lack of time and resources through an extremely busy ~18 month period, and that we aim to be more timely in releasing these voluntary progress reports in the future. Anyway, for me it is a notable report, as it was my first year of formal involvement on the project; you’ll see mention of my work, alongside the strides made by the team and project as a whole.

---

The Social Web: Decentralization and Building a Better Internet

Hello everyone.

I am Andy Piper. You may know me as a developer advocate on the internet. I’ve worked in a few places during my career, usually talking about messaging APIs and interoperability. Right now, I am a freelance technologist, and I continue to care about the Web and Open Source technologies, as I always have done.

One of the ways that I contribute to the web and open source in practical terms, is that I work with the Mastodon team on developer relations and community.

It is wonderful to be here!

This is not my first time in Japan, but it has been a few years since I was last able to visit this wonderful country. Thank you for having me here again. It is extra exciting for me, because it is Ruby Week! I’m honoured to be here in Shimane, speaking to such a talented group of Ruby developers.

I’ve used a number of different programming languages in my career, some a lot more than others. Let me tell you about my relationship with Ruby.

My first experiences with Ruby were back in 2012, when I worked on the early version of Cloud Foundry at VMware and Pivotal. Cloud Foundry eventually switched to Go, but I spent a lot of time with Ruby back then. I have also had the pleasure to meet Matz a couple of times before on my previous visits to Japan. I do not use Ruby myself very often today, but I still appreciate its value and importance, and I will talk about that today.

Why am *I* here? Well, I am very happy and honoured to be here on behalf of Eugen Rochko, the founder of Mastodon, who was not able to travel here to speak to you all in person.

I have a short message from Eugen that I brought with me, and I’d like to play that for you now.

[EUGEN VIDEO]

Hello everyone. My name is Eugen Rochko, I am the founder and current CEO of Mastodon.

Thank you for inviting me to RubyWorld, I hope that you have a great conference.

I believe that social media should not belong to a single company that controls and owns everything that you post online, and this is why I founded Mastodon, which is built on the principle of protocols over platforms.

Mastodon is built on the ActivityPub protocol and is part of the Fediverse, which is a network of interoperable social media platforms, that include many different open source software projects, like Misskey, Pleroma, Pixelfed, and so on; as well as commercial platforms like Flipboard, Threads, and hopefully many more in the future.

Users from all of these platforms can communicate with each other seamlessly as if they are on a single network, but at the same time control over the network does not belong to anyone in particular, and everyone has control over their own part of it. This allows different approaches to social media, to user experience, to business models to sort of coexist together, it allows limitless experimentation and it gives developers the security that they can build on these standard APIs without the risk of anyone shutting them down.

Mastodon itself is built on principles of respect for users choice. I think that nowadays what you often find is that platforms only offer you two options: “yes”, and “maybe later”, and I think that is emblematic of the way that Big Tech no longer treats users as adults. In Mastodon you curate your own home feed – you follow people, you follow hashtags, and then you see them in your home feed, no surprises. When you search for something, we give you results for your search, we don’t try to guess what else you might have meant – in the current landscape of social media I think it is quite a unique spin on things where you still feel like you’re in control.

Building a social media platform to compete with Facebook and Twitter who have thousands and thousands of engineers, is not easy, but it is a little bit easier with Ruby on Rails, because Ruby is a very beautiful and expressive language that is just very pleasant to work in. And it definitely helps that with Ruby on Rails you have conventions of where things are, and it is very easy for new developers to get started with our project because they know where things are, there is a familiar layout. I’ve been a Ruby developer for a long time and it is my favourite language. It is certainly also helpful that the ecosystem of Ruby Gems can provide so many different options for functionality that you may not want to build in-house. One of the recent developments in the Ruby language that I haven’t had the chance to try out yet but I’m sort of curious about is static type checking because it’s something that I’m getting used to with TypeScript and I find it very useful, and I know that there has been movement in this regard in the Ruby ecosystem, so eventually hopefully we will get around to it in Mastodon; but for now it hasn’t been the case.

The whole decentalised social web ecosystem has seen amazing growth in recent years and this is what Andy is here to talk about. I think it is more important now than ever to build a truly decentralised social web where control is in the hands of the people and not just one or two US corporations.

Thank you very much. I know that Mastodon is popular in Japan, it’s one of the first places where Mastodon became popular, I really appreciate your support.

And, have an amazing conference![/END]

So let me begin by telling you what we’re going to cover today.

– First, I’ll talk about the decentralized social web, and why that’s important; right now, more than ever!

– We will look into how the Social Web has grown.

– We will talk a little bit about why it is difficult to make something better, but why it is worth doing.

– and of course, I want to talk about Ruby as well!

And I hope to convince you all that the Social Web needs your passion, code, and support.

Let’s get started.

[section 1]

Today I’m excited to talk about how WE – all of us – can build a better internet, one that centres on users and communities, not corporations with hidden motives.

I’m here to represent Mastodon, this decentralized social media platform that’s part of the larger, growing open Social Web, and that is written in Ruby. I also care about making a better online experience for everyone, beyond Mastodon!

In order to understand why decentralization matters today, let’s look back at how the internet has developed.

My university degree subject was History, and I believe that we need to understand the history of our technology. I think that my interest in history has also given me a deep interest in people, society, and culture.

The internet started as a space for collaboration.

The World Wide Web offered *incredible* opportunities for people to share, connect, and create freely. It was meant to be a space of open exchange. Sir Tim Berners-Lee, the inventor of the web who is from the UK like me, famously said: “This is for everyone”.

The *PROBLEM* was that as the web grew, so did the power of a small number of technology giants. Today, a few dominant companies set the rules for most online social spaces.

Society and technology are extremely interconnected. We see this all the time.

We thought we were building a digital public square, but that evolved into a collection of private, profit-driven platforms that prioritize data collection for advertising, and algorithms to drive engagement over user well-being.

You might notice something else, as well – those private platforms don’t want you to easily access and share content between them, they want you to post your content into, and stay on, their platforms! Very often, they do not want you to make that choice to use a different tool, so they add more and more features into their own apps like chat, and video, and selling you things, so that you never have to leave.

Around the world, Right Now – we see changes that might challenge the freedoms that the internet has shown us all.

With so much power centralized by technology giants that are either in the USA or China, we’ve seen them influence decisions affecting our privacy, data security, and even democracy itself.

[section 2]

Mastodon offers an alternative to the closed social platforms.

It is based on the web’s open, collaborative roots. It’s part of the Fediverse (or Social Web), a collection of interconnected services that operate independently, but can communicate seamlessly – this is called “federation”.

Mastodon and the Social Web are powered by the ActivityPub protocol, and over the past couple of years there has been rapid growth in the number of services embracing federation.

ActivityPub is a standard that is overseen by the W3C, the same organisation that supports many of the core web standards.

Recently, the Social Web Foundation was formed, to help support and sustain momentum around ActivityPub. In their launch post, they wrote:

“The “social web”, also called the “Fediverse”, is a network of independent social platforms connected with the open standard protocol ActivityPub. Users on any platform can follow their friends, family, influencers, or brands on any other participating network.

ActivityPub was standardized by the World Wide Web Consortium (W3C) in 2018. It has attracted OVER ONE HUNDRED software implementations, tens of thousands of supporting web sites, and tens of millions of users.”

This is not just about adopting a technical protocol, although ActivityPub is very important.

• user-centric, privacy, locality, choice, resilience

This is a philosophy that puts users first, emphasizing local and community control and user empowerment. It is about an internet culture that values privacy, locality, choice, and resilience.

These are the principles that define Mastodon and the Fediverse.

When we think of social media, we often talk about that small group of big companies, and those dominant and centralised platforms who don’t really want you to go or to connect anywhere else.

The Fediverse changes this. Anyone can run their own platform, build their own community, and still communicate with others around the world.

This is also really important for public institutions such as governments and other organisations.

They have the opportunity to run their own services. This means that they no longer have to rely on private companies owned by billionaires, who can limit their access or ability to post at any time. I wrote about this on my blog several times this year. We see a number of organisations actively operating their own social web presences today, particularly in Europe.

In the light of recent global developments, I think we will see more countries and organisations realising that having all of your data hosted by a few US-based companies is not necessarily a great idea.

Mastodon is one of the most well-known fediverse platforms, but there are services like PeerTube for video sharing, Pixelfed for photos, BookWyrm for reading.

There is a lot of opportunity to build new things. There is a new short video app called Loops (a bit like TikTok) which is part of the movement to federate as well.

There are existing networks opening up, as well.

In the last year, we have seen Meta’s Threads platform launch with a promise that it will be part of the fediverse. It has slowly started to federate, and users on other non-Meta platforms can follow Threads users if they opt-in to sharing. Just a few hours ago today, Threads enabled the ability for users on Threads to follow users on other platforms, so for the first time you can follow my Mastodon account (which is my main social account) on Threads! This is very very cool!

We have also seen established platforms like Flipboard and WordPress join the fediverse. The blog and newsletter platform Ghost is posting regular updates as they start to add features for federation as well.

All of this progress is really cool – it means I can write my blog on WordPress, post my photos on Pixelfed, keep my videos on PeerTube, and I can share those same things directly on my Mastodon account – or any platform that supports ActivityPub.

One of the largest Mastodon communities is right here, in Japan.

This dates back to 2017, right after Mastodon was started. mstdn.jp was created in April 2017 – it went viral and saw a growth of tens of thousands accounts within a few days, a catalyst for Mastodon’s popularity in Japan.

If you look at this, you can see that mstdn.jp is still the second largest Mastodon instance worldwide, by numbers – both overall registered users, and the monthly active user count.

We really want to say thank you to our friends at Sujitech for running the instance, and for their support of Mastodon.

Mastodon is not only about individual large instances, though – it is important that there is a diverse community of different servers, with their own rules and owners, to accommodate many different interests. Diversity makes the internet stronger, and it makes the web more interesting.

Of course, Mastodon itself is only part of the fediverse story – for example, I know that the Misskey community is very active in Japan as well. I think the interest in Mastodon and the fediverse in Japan might be because they have values that are understood here: local community, respect for users, and data sovereignty.

[section 3]

Building a decentralized social network like Mastodon has a unique set of challenges, both technical and social. I want to spend a bit of time talking about those, because we need to make an effort together to overcome them, if we want to build a better system together.

A centralized platform has one set of rules, one server architecture, and one approach to moderation. Oh, and usually there’s only one owner with one business model, as well.

That makes it mostly easy to manage, but means that it can be difficult to fit well with a local culture, or to support a range of different opinions and views.

A centralized platform can also remove access or choose to limit the ability to post.

In a federated system, there is no central control. Every server and platform may have its own rules, and be run in a different way, under different legal requirements.

Mastodon (and other ActivityPub services) operates as a federation of independently hosted servers, or “instances.” Each instance needs to handle its own user data, security, and interactions with other servers. There are technical challenges to hosting and scaling the data, and the network connections. We need to discuss and agree on the technical standards, as well. The Social Web Incubator Community Group at the W3C, and the less formal Fediverse Enhancement Proposals at SocialHub, are both parts of this discussion.

There are also *social* challenges. The responsibility for content moderation and community standards belongs to the individual instances and their administrators. This can be great! It means that communities set their own standards, with a more localized experience. The challenge is that it also means that we need to provide good moderation tools, and to build strong ways to collaborate between instances.

People can be as complicated as technology!

If you are interested in these topics – moderation and user safety – there is a non-profit organization, called IFTAS – Independent Federated Trust and Safety – that is working on creating new tools and education in this area. It is an important thing to understand.

This is the internet, and the internet connects our world.

This means that we must THINK truly globally. We need to enable people in different countries to use the Social Web in ways that make cultural sense. We need people from places such as Japan and other countries in this region to take part in the conversations about technology standards, and user expectations.

This is difficult. There are language barriers between us, and timezones, and sometimes it can be a bit scary when we feel less confident about how we communicate.

There is a lot of work to do, and there are a lot of tools to build, to make the decentralized Social Web successful, and to make it as accessible to as many people as we want it to reach.

[section 4]

This brings me to my final section – the role of developers, particularly Ruby developers – in building the Social Web.

The Ruby community is global and thriving, and that is very exciting and empowering. In my role as a developer advocate, as someone who works with developer communities, I’ve seen how Ruby’s reach and culture have enabled it to grow and succeed.

Here is the *cool* part – Ruby has been a part of this story for at least as long as Mastodon itself, because Mastodon is built using Ruby and Ruby on Rails. And it is going strong!

Mastodon is not the only ActivityPub service built with Ruby. For example, there is a new app, called ManyFold – a 3D printing catalog app – that is also built with Rails. There is also a Ruby gem called Federails that provides ActivityPub support for apps, and I know that the author of ManyFold has been contributing to that. If you look at the “delightful Fediverse” list of apps, there are at least 20 references to Ruby, and that is really cool!

And of course you have your own “Matz-todon” instance, ruby.social!

I’m here to talk about Mastodon! So let me tell you a bit about the importance of Ruby to the Mastodon project.

Six months ago, we started to share a monthly blog series which is “behind the scenes” from our development team, where you can follow what we are working on. We call this “Trunk and Tidbits”. If you read these blog posts, you will see a summary of the changes we are making over time, and also learn about new things that we are working on.

I help to write the blog posts, but I told you at the start of the talk that I do not write and run Ruby code myself every day, so I talked to the team to get a good understanding of what we have been building and learning!

In the most recent release, 4.3 that came out in October this year, we updated our dependencies. We support the maintained Ruby versions, and try to update our recommended version to the most recently-released Ruby. With 4.3, we moved our baseline to Ruby 3.3.

We saw significant improvements after updating to Ruby 3.3 and YJIT.

When we upgraded our flagship instance – mastodon.social – from Ruby 3.2 (with YJIT enabled) to Ruby 3.3 also with YJIT, we saw 15-20% response time and CPU usage improvements. We really love these kinds of improvements in Ruby, and we are already planning to move on to Ruby 3.4 in the future.

Our backend stack is a classic Rails app with Sidekiq and Postgres, in a big monolith. We serve more than 150k requests per minute at peak on mastodon.social, with a 120ms P90 response time. We process more than 200 million Sidekiq jobs per day, on 160 CPU cores, which is quite impressive!

By the way, as well as mastodon.social, we also operate a second instance, mastodon.online. We run the latest code from GitHub on mastodon.social and mastodon.online as much as we can, to get new features to a lot of users and to get a good idea of the performance and usability of our updates.

We try to use modern Ruby constructs where we can. The Mastodon codebase is very clean and well organised for such a large project, thanks to the expressiveness of Ruby, and Rails patterns. We find that this combination makes it easy to prototype new features as we improve Mastodon.

As well as the overall Ruby runtime, we’re also closely following the recent performance improvements in the json gem. Right now, we use a third-party gem for JSON processing called oj – this has been valuable for many years, but the recent work on the json gem has brought it level with oj, and our preference to reduce dependencies as much as possible means that we hope to move to the json gem soon.

A lot of this great performance work originates from Shopify, so we are really grateful to their contributions to Ruby! As a community, please continue to improve Ruby, and make it better for everyone!

There are a number of ways for Ruby developers to get involved with Mastodon itself. The best place to start is our CONTRIBUTING guide, which has a lot of information about how our small team works, and how we accept contributions from the developer community.

My role on the team is to help developers to work with the Mastodon API, and to be successful building apps with Mastodon. Our team is in different places around the world, but we might not always know how users in countries such as Japan prefer to use a social network, so I strongly believe that it is important to have a variety of apps available. We know that there are some great apps from here in Japan, such as SoraSNS on the iPhone.

I want to finish my talk today, with this message.

It’s not just about Mastodon. It’s about you, and it’s about all of us.

You – Ruby developers – have an opportunity to help to expand the fediverse and open social web ecosystem, by building new libraries, apps, and tools, even whole new platforms. Go out and invent new things!

The mayor said this morning: “Open-mindedness and communication enrich the community and give shape to dreams.”

Wonderful.

The website describes Ruby Week as: “a campaign that expresses the values of “freedom, fun, and contribution to the community” that are alive in Ruby.” and I love that description! Freedom, fun, and contribution. This is great.

It is not just writing code.

We – all of us here in this room today – are able to shape the future of the internet with all of our contributions to the community.

We – ALL of us here in this room today – need to tell the story of the web to our friends and families, opt when we can to support open platforms over closed algorithms, and we need to make sure that the future of the internet is decentralized, safe from efforts to subvert our freedoms.

We have the power to build an internet that is open, free, and based on the values of community and trust that we care about.

I’m here today and asking you all to help us on the Mastodon team, and friends of the Social Web Foundation, to do that.

Finally – again, Mastodon is a non-profit organization. We rely on donations to keep us going.

Please support us if you can. It is more important than ever before that there are financially stable, sustainable, open alternatives to the legacy centralized platforms operated out of one or two specific countries.

We appreciate all of you, and I want to thank the developer community here in Japan and the Ruby developer community, for everything you bring to the free and open Social Web.

I have stickers to share with you to say thank you.

Thank you! Enjoy RubyWorld!

---

I should have done more research when I spoke about Mastodon client apps in my #RubyWorld talk, I mentioned that SoraSNS is made in Japan… of course, so are Feather, Subway Tooter, Dawn, ZonePane, TootDesk and maybe even more that I missed mentioning. I love the diverse range of apps that are available for different cultures and preferences! Thank you to the developer community in Japan! -> joinmastodon.org/apps

— Andy Piper (@andypiper@macaw.social) 2024-12-06T10:10:56.516Z

(in retrospect, I should definitely also have added that we are also using Ruby for the new Fediverse Discovery Providers project, aka “Fediscovery”, that was initiated at Mastodon but is independent and open to the broader fediverse – ready for some interoperability testing).

---

1. This was the ability to follow fediverse users from Threads; although, no replies or interaction as yet, and there’s still a lot for them to do to be full and fair participants in the fediverse as a whole ↩︎

Share this post from your fediverse server
https:// Share

This server does not support sharing. Please visit .

andypiper.co.uk/2024/12/24/bui…

#100DaysToOffload #activitypub #andyPiper #fediverse #Japan #mastodon #presentations #publicSpeaking #ruby #rubyworld #socialWeb #speaking #talks #travel

The title of the keynote at RubyWorld Conference 2024 has been decided

The title of the keynote at RubyWorld Conference 2024 has been decided.

^{RubyWorld Conference 2024}