Lemmy is a link aggregator / Reddit clone for the fediverse (right from the repo’s readme).

Uhhh, okay?

On Lemmy a user can post links, pictures or just text. The published content can then be commented and upvoted or downvoted by other users.

Content is organized by communities and each post is linked to a community. Anyone can subscribe to any community they find interesting and see the matching content on their homepage. Any filter can be applied, whether we want to see subscribed content or not, sorted by comments, date or activity, …​

Up to this point, this is exactly the same as Reddit. The interesting part (well that depends on who you ask but in my world it is indeed interesting!) is that Lemmy is a free/libre/open-source software (AGPL-3.0), allowing us - amongst other things - to selfhost it.

What’s even better is that instances can be federated, so that I can access the content of any federated instance from mine. I would even say that it’s vital in my case since there’s a good chance I will be the only user on my instance, the experience would be quite dull in the absence of federation…​

Note that ActivityPub is the protocol used to federate, which is the same used by Mastodon and since the last update (three days ago as I publish this) we can follow Lemmy communities from Mastodon and Pleroma.

As I always say about my self-hosting adventures, because I can and because I want to do it. I like the idea of a decentralized Internet and I like administrating servers and services so one could rather ask: why not?

Well, there were a few concerns at first, such as cost. Since I already have servers at my disposal for other services, it doesn’t cost me anything extra.

It could also be time consuming to manage all those services, that brings me to the how. As I already said here, to mitigate this issue I use packaged software anytime I can. In that regard I have a YuNoHost instance and you guessed it, someone had already packaged Lemmy for YuNoHost.

As it can hapen with some YNH packages, this one was not up-to-date and did not work perfectly (I know what I’m talking about, the restic package I created is not up-to-date and does not work perfectly, far from it). No matter, I saw that someone else had forked the project and had tried and failed to update it. Still no matter, I managed to fix it.

I was not sure about the way I fixed it though so I asked about it on the PR, I’ll probably make my own PR if someone answers me.

In the mean time a new version had been released (the one with Mastodon and Pleroma federation), so I updated the package again and deployed the new version on my instance. Everything is running fine.

Up to this point, I had a working up-to-date federated lemmy instance. Everything was working perfectly.

Well, almost perfectly…​

By default, my instance could only list local content. Remote communities only get listed once a local user has search for them, furthermore if they have subscribed.

The issue with this approach is how can I discover new communities to subscribe to?

On Mastodon one "subscribes to" users (rather follows), which allows them to see those users' threads involving other users which can in turned be followed. On Lemmy, we subscribe to communities, and in this case other communities are almost never brought up naturally.

The only thing I could do was go on lemmy.ml, list all publications, look for interesting communities, search for them on my instance and then subscribe. Quite tedious…​

But there was a solution: I knew that searching for a federated community from my instance would get it listed on mine. Why not search for all of the communities of a federated instance so that they all get listed on mine?

So I came up with a little script. I talked about it with an admin of lemmy.ml beforehand as I did not want to do something wrong and somehow get my instance blacklisted. They said it was ok, as long as I don’t DDOS their instance. I think I’ll be okay, I don’t want that, neither could I do it.

Here’s the help message from the script:

The script has two dependencies: BeautifulSoup and requests which can be easily installed with a pip install bs4 requests (although I rather use virtualenvs)

It lists all communities from the source instance, does a search on the destination instance for each one and stores it in a cache.txt file in the current directory so that we can ignore them next time the script is ran.

The script will only process a defined number of new communities (COMMUNITIES_BATCH_SIZE) at each run, which defaults to 15.

The only thing left to do is to schedule it at first every fifteen minutes for example, until all communities are retrieved. Then only once a day, or even once every few days should be enough, I don’t think there are a lot of new communities created everyday.

Here’s what I put in my crontab on a server for a daily execution a 3:29 PM

For now it’s working as I imagined it would, I still have to check if the retrieved communities don’t get somehow deleted if they are never used (if nobody subscribes). If that’s the case I would have to come up with a solution but I’ll worry about that when and if the issue arises. Apparently communities discovered this way never get deleted, awesome!

As a reminder I’m not a developper, there must be a lot of room for improvement in the script but it works for me, that’s enough.

I wanted to leverage the Lemmy API at first but I could never understand how to use it. Must be my lack of developing skills :p

If this script is useful to you of if you improve it in any way let me know!

You can reach me:

You can also discuss it on https://lemmy.coupou.fr/post/10428