Scrapping telegram channels with python

In this short tutorial, I want to show how to access telegram using Telethon python library. We will use Python 3.9 to achieve it. First, we will connect to a channel and get all the messages from there.

python3 -m pip install --upgrade telethon

To connect to a channel, you need to obtain API hash and app id from https://my.telegram.org/apps. Remember not to share them with anyone!

Let us create a client for our usage.

client = TelegramClient('me', <YOUR_APP_ID>, <YOUR_APP_HASH>)
await client.start()

This will create a client and connect it to the telegram network. The first argument of the client constructor is the session name. Telethon will automatically create a session file at the root of your project. The file will be named after the session name and will have a .session extension. In our case, it will be me.session. Remember not to share this file with anyone.

Telethon will ask you to provide your phone number and security code from telegram on your first connection. After that, if your session stays the same, you won’t have to give those details again.

The client is using async io from python 3.7. If you couldn’t take a look at this concept, please check this tutorial.

We don’t have to connect and wait manually. Moreover, the client comes with support for context manages, which allows us to use it with with statement like this:

with client:
    #do something with the client. Python will take care of stating and stopping

Let us try to connect to the python developers channel – t.me/Python. Even though this link is enough for your desktop client to open this channel, we need something more in telethon – the telegram entity. To get it, we will make another call to the telegram network.

channel = await client.get_entity("t.me/Python")

As a result, we receive the object, using which we can connect to a channel and download messages from there.

async for message in client.iter_messages(channel):
    print(message)

Getting all the messages can be done using an iterator exposed by the client instance. We just need to pass the channel object obtained in the previous step. After that, Telethon takes care of getting all the messages, chunking requests, making sure that we don’t break telegram API limits, etc. By default, messages are returned from the newest to the oldest. You can reverse this sorting by passing reverse=True to the iter_messages method.

This method also has many filtering possibilities. For example, you can filter by the sender, look only for messages with photos or match messages by fuzzy text. Check the iter_messages documentation for more details.

The message object obtained this way is very rich. Apart from the unique id and contents, we can get sender, attached media, and even bot id (if the message was sent by a bot). For more details, please check the message documentation.

Last but not least. Telethon’s client comes with a very convenient asyncio loop that can be accessed with client.loop call. We can use it to run until canceled or, more interesting in our case, run until all the messages are downloaded from the channel that interests us. The complete code example looks like this:

channel_link = "t.me/Python"

client = TelegramClient('me', api_id, api_hash)


async def main():
    channel = await client.get_entity(channel_link)

    async for message in client.iter_messages(channel):
        print(message)


if __name__ == '__main__':
    with client:
        client.loop.run_until_complete(main())

In my opinion, Telethon is an excellent library. It takes a lot of burden from the user (authentication, rate limiting, batching, etc.) and hides all this complexity behind a very readable API. I encourage you to try it yourself and see how great it is!