Telegram bot as a first project and its significance for professional growth based on personal experience

Greetings to all! Tell us about yourself. I am 24 years old, graduated from a technical university last year and still have no work experience. Looking ahead, I want to say that initially, in the laid down plan (drawn up in the fall of 2019), I planned to go to work in March-April 2020, but, unfortunately, quarantine intervened, so I postponed everything until mid-summer and in the future I hope to write my own success story. I was never drawn to programming. At the university they taught enough programming, but this craft could not interest me then. There were also procedural languages ​​(C), a year-long course in OOP (Java), databases, even assembler and C++. But to be honest, I was generally indifferent to studying, since most of the disciplines taught seemed useless to me, suitable only for reporting purposes (in principle, this is so). After graduating from university, I had to make a decision: I didn’t acquire some skills, but I needed to work. I had to think about self-education (oh, I’ve already missed at least 2 full years by sitting idly by) and the choice naturally fell on Java, since in an OOP course at the university one of the guys recommended the javarush course, and he, as you know , is dedicated specifically to the Java language. I was interested in the presentation of the course. Yes, I didn’t like programming then, because I immediately gave up when I encountered any difficulty, and there are more than enough difficulties in programming. But at the same time, I felt that I wanted to write code, so in the end I decided to get into programming. I’ll briefly tell you about my experience with javarush. I started in August 2019, immediately bought a subscription for a month, but at level 7 I realized that the tasks were difficult. I put aside the course and picked up Shildt. So in parallel I completed the course for 3 months. I reached level 20 (this is my second account), read Schildt almost completely, then got tired of the tasks here, in which I stopped seeing practical benefits for myself. I went to codewars, leetcode, and started watching video courses. By the way, in 3 months I went from “Oh no, what is an array? How to work with it and why is it so scary”? to a detailed study of the source code of collection classes (ArrayList, HashMap, etc.). Based on personal experience, I will tell beginners: the main thing here is to overcome the feeling that arises if you don’t understand anything and can’t solve anything. When it arises, you just want to give up everything and it seems that you are too stupid for this matter. If you overcome such moments within yourself and mentally rest, then success will come. I think many people can’t cope with this, so they quickly give up on such endeavors. As a result, in December 2019 I started thinking about my project. I decided to choose a Telegram bot, but there was no idea. At the same time, one friend needed functionality for his group in a telegram, which he would like to automate. He was just aware that I was studying programming in depth and offered me a project. For me, for experience and a future resume, for him, for the development of the group. I will even allow myself to quote his idea: "Недавно софтину хотел у программиста заказать, которая загружала бы в выбранное Облако файлы по прямым linkм. Это интересно, так How аналогов нет. И просто очень удобно. Суть: копируешь ссылку, вставляешь в окно и выбираешь нужное Облако (GDrive, Mail, Яндекс Диск и т.п), в своё время софт всё делает на стороне serverа и юзеру ничего не нужно загружать на свою машину (особенно круто, когда у тебя сборка на SSD-накопителях). Думали сделать в web-интерфейсе, чтобы можно было запускать How с телефонов, так и с десктопа... Можно в принципе через приложение реализовать, а не через web-интерфейс. Тебе такое по силам?"I started working, but in the end, after a couple of days, I realized that nothing would work out for us, largely due to a lack of knowledge. A friend needed these same links to Cloud.Mail, but they still don’t have an API. There was an attempt to put something together via GDrive, but the implementation was lame, plus this cloud service did not suit the “customer.” Although initially he offered several clouds to choose from, he ultimately rejected everything except mail.ru, for which no solution was found. somehow it all turned out to be expensive, it was necessary to connect the database, use a server for storage, etc. By the way, it still needs this web application. Since things didn’t work out for us, I decided to make an information bot. It should was to receive links to the game from the Google Play store, parse the link and save the received information to the library, and then write it to a json file. Thus, with each request, the library can expand thanks to the efforts of users. In the future, you can not get information about the game in a convenient form by going to Google Play. You just write the command /libraryHere_game_name and get everything you need. But there are several difficulties that I will tell you about later. At first I progressed slowly, as I started taking two SQL courses at the same time. I simply couldn’t understand how the bot worked at all and how to process requests. I met a friend who was also interested in working on the project. The first version of the bot was ready in about a month, but disagreements arose with a friend (on my part). I took up the part of the bot that is responsible for parsing, and he directly worked on requests to the bot and their processing. For some reason, he began to complicate the bot, introduce some kind of authorization, invent admins, add unnecessary functionality, plus I didn’t really like his coding style. In my opinion, this was not necessary in an information bot. So I decided that I would write a bot from scratch myself with the functionality I needed. Now I’ll tell you what the bot actually does (using an example from the project code). I will attach the full code of the project at the end of the article and, unfortunately, I will not be able to fully comment on it. Any user message sent to the bot is an object of the Update class. It contains a lot of information (message id, chat id, unique user id, etc.). There are several types of update: it can be a text message, it can be a response from the telegram keyboard (callback), a photo, audio, etc. To prevent the user from messing around too much, I process only text requests and callbacks from the keyboard. If the user sends a photo, the bot will notify him that he does not intend to do anything with it. In the main bot class, in the onUpdateReceived method, the bot receives an update.

@Override
    public void onUpdateReceived(Update update) {
        UpdatesReceiver.handleUpdates(update);
    }

which I pass to the handler (own UpdatesReceiver class):

public static void handleUpdates(Update update) {
        ...
        if (update.hasMessage() && update.getMessage().hasText()){
            log.info("[Update (id {}) типа \"Текстовое сообщение\"]", update.getUpdateId());
            new TextMessageHandler(update, replyGenerator).handleTextMessage();
        }
        else if (update.hasCallbackQuery()) {
            //логгирование
            new CallbackQueryHandler(update, replyGenerator).handleCallBackQuery();
        }
        else {
           //логгирование
            replyGenerator.sendTextMessage(update.getMessage().getChatId(), "Я могу принимать только текстовые messages!");
        }
    }

UpdatesReceiver is a central handler that, depending on the type of update, transfers control to another specialized handler: TextMessageHandler or CallbackQueryHandler, to whose constructors I pass update further down the chain. Update is the most important thing when working with a bot and cannot be lost, because with the help of the information stored in the update, we find out which user and to which chat the response should be sent. To generate responses to the user, I wrote a separate class. It can send regular text message, message with inline keyboard, message with picture and message with reply keyboard. An inline keyboard looks like this: It defines buttons that, by clicking on them, the user sends a callback to the server, which can be processed in almost the same way as regular messages. To “maintain” it you need your own handler. We set an action for each button, which is then written to the Update object. Those. for the "Cost" button we set the description "/price" for the callback, which we can later get from the update. Next, in a separate class, I can already process this callback:

public void handleCallBackQuery() {
  String call_data = update.getCallbackQuery().getData();
  long message_id = update.getCallbackQuery().getMessage().getMessageId();
  long chat_id = update.getCallbackQuery().getMessage().getChatId();
    switch (call_date){
      case "/price" :
        //тут что-то сделать
        break;
...

The Reply keyboard looks like this: And in essence, it replaces the user's typing. Clicking the "Library" button will quickly send a "Library" message to the bot. For each type of keyboard, I wrote my own class, implementing the Builder pattern: inline and reply . As a result, you can essentially “draw” the desired keyboard depending on your requirements. This is terribly convenient, since keyboards may be different, but the principle remains the same. Here is an intuitive method to send a message with an inline keyboard:

public synchronized void sendInlineKeyboardMessage(long chat_id, String gameTitle) {
        SendMessage keyboard = InlineKeyboardMarkupBuilder.create(chat_id)
                .setText("Вы может узнать следующую информацию об игре " + gameTitle)
                .row()
                .button("Стоимость " + "\uD83D\uDCB0", "/price " + gameTitle)
                .button("Обновлено " + "\uD83D\uDDD3", "/updated " + gameTitle)
                .button("Версия " + "\uD83D\uDEE0", "/version " + gameTitle)
                .endRow()
                .row()
                .button("Требования " + "\uD83D\uDCF5", "/requirements " + gameTitle)
                .button("Покупки " + "\uD83D\uDED2", "/iap " + gameTitle)
                .button("Размер " + "\uD83D\uDD0E", "/size " + gameTitle)
                .endRow()
                .row()
                .button("Получить всю информацию об игре" + "\uD83D\uDD79", "/all " + gameTitle)
                .endRow()
                .row()
                .button("Скрыть клавиатуру", "close")
                .endRow()
                .build();
        try {
            execute(keyboard);
        } catch (TelegramApiException e) {
            log.error("[Не удалось отправить сообщение с -inline- клавиатурой]: {}", e.getMessage());
        }
    }

To give the bot strict functionality, special commands using the slash character were invented: /library, /help, /game, etc. Otherwise, we would have to process any garbage that the user might write. Actually, this is what MessageHandler was written for:

if (message.equals(ChatCommands.START.getDescription())) {
     replyGenerator.sendTextMessage(chat_id, new StartMessageHandler().reply());
     replyGenerator.sendReplyKeyboardMessage(chat_id);
}
else if (message.equals(ChatCommands.HELP.getDescription())
             || message.equalsIgnoreCase("Помощь")) {
      replyGenerator.sendTextMessage(chat_id, new HelpMessageHandler().reply());
}
 ...

Thus, depending on what command you send to the bot, a special handler will be included in the work. Let's go further and look at the work of the parser and library. If you send the bot a link to a game in the Google Play store, a special handler will automatically work . In response, the user will receive information about the game in the following form: At the same time, a method will be called that will try to add the game to the bot’s library (first to the local map, then to -> json file). If the game is already in the library, then a check will be carried out (as in a regular hashmap), and if the field data (for example, the version number has changed), then the game in the library will be overwritten. If no changes are detected, then no entries will be made. If there was no game in the library at all, then it is first written to the local map (an object like tyk ), and then written to a json file, since if the application on the server is unexpectedly closed, the data will be lost, but it can always be read using the file. Actually, when the program starts, the library is always loaded for the first time from a file from a static block:

static {
        TypeFactory typeFactory = mapper.getTypeFactory();
        MapType mapType = typeFactory.constructMapType(ConcurrentSkipListMap.class, String.class, GooglePlayGame.class);

        try {
            Path path = Paths.get(LIBRARY_PATH);
            if (!Files.exists(path)) {
                Files.createDirectories(path.getParent());
                Files.createFile(path);
                log.info("[Файл библиотеки создан]");
            }
            else {
                ConcurrentMap<string, googleplaygame=""> temporary = mapper.readValue(new File(LIBRARY_PATH), mapType);
                games.putAll(temporary);
                log.info("[Количество игр в загруженной библиотеке] = " + games.size());
            }
        }
        catch (IOException e) {
            log.error("[Ошибка при чтении/записи file] {}", e.getMessage());
        }
    }

Here you additionally have to read data from the file into a temporary map, which is then “copied” into a full map in order to maintain case insensitivity when searching for a game in the file (by writing tITan QuEST, the bot will still find the game Titan Quest in the library). It was not possible to find another solution, these are the features of deserialization using Jackson. So, with each request for a link, the game is added to the library, if possible, and the library thereby expands. Further information about a specific game can be obtained using the command /libraryGame_Name. You can find out both a specific parameter (for example, the current version) and all parameters at once. This is implemented using the inline keyboard, which was discussed earlier. During the work, I also applied the skills acquired here while solving problems. For example, a list of names of random games located in the library (the option is available using the /library command):

private String getRandomTitles(){
        if (LibraryService.getLibrary().size() < 10){
            return String.join("\n", LibraryService.getLibrary().keySet());
        }
        List<string> keys = new ArrayList<>(LibraryService.getLibrary().keySet());
        Collections.shuffle(keys);
        List<string> randomKeys = keys.subList(0, 10);
        return String.join("\n", randomKeys);
    }

How does the bot process links? It checks them to see if they belong to Google Play (host, protocol, port):

private static class GooglePlayCorrectURL {

        private static final String VALID_HOST = "play.google.com";

        private static final String VALID_PROTOCOL = "https";

        private static final int VALID_PORT = -1;

        private static boolean isLinkValid(URI link) {
            return (isHostExist(link) && isProtocolExist(link) && link.getPort() == VALID_PORT);
        }

        private static boolean isProtocolExist(URI link) {
            if (link.getScheme() != null) {
                return link.getScheme().equals(VALID_PROTOCOL);
            }
            else {
                return false;
            }
        }

        private static boolean isHostExist(URI link) {
            if (link.getHost() != null) {
                return link.getHost().equals(VALID_HOST);
            }
            else {
                return false;
            }
        }

If everything is in order, then the bot connects via a link using the Jsoup library, which allows you to get the HTML code of the page, which is subject to further analysis and parsing. You won't be able to fool the bot with an incorrect or harmful link.

if (GooglePlayCorrectURL.isLinkValid(link)){
     if (!link.getPath().contains("apps")){
         throw new InvalidGooglePlayLinkException("К сожалению, бот работает исключительно с играми. Введите другую ссылку.");
     }
     URL = forceToRusLocalization(URL);
     document = Jsoup.connect(URL).get();
 }
     else {
         throw new NotGooglePlayLinkException();
      }
...

Here we had to solve a problem with regional settings. The bot connects to the Google Play store from a server located in Europe, so the page in the Google Play store opens in the appropriate language. I had to write a crutch that forcibly “redirects” to the Russian version of the page (the project was, after all, aimed at our audience). To do this, at the end of the link you need to carefully add the parameter hl: &hl=ru in the GET request to the Google Play server .

private String forceToRusLocalization(String URL) {
        if (URL.endsWith("&hl=ru")){
            return URL;
        }
        else {
            if (URL.contains("&hl=")){
                URL = URL.replace(
                        URL.substring(URL.length()-"&hl=ru".length()), "&hl=ru");
            }
            else {
                URL += "&hl=ru";
            }
        }
        return URL;
    }

After a successful connection, we receive an HTML document ready for analysis and parsing, but this is beyond the scope of this article. The parser code is here . The parser itself retrieves the necessary information and creates an object with the game, which is later added to the library if necessary. <h2>To summarize</h2>The bot supports several commands that contain certain functionality. It receives messages from the user and matches them with its commands. If it's a link or the /game + link command, it checks that link to see if it belongs to Google Play. If the link is correct, it connects via Jsoup and receives the HTML document. This document is analyzed based on the written parser. The necessary information about the game is extracted from the document, and then the object with the game is filled with this data. Next, the object with the game is placed in local storage (if the game is not there yet) and immediately written to a file to avoid data loss. A game recorded in the library (the name of the game is the key for the map, the object with the game is the value for the map) can be obtained using the command /library Game_name. If the specified game is found in the bot's library, the user will be returned an inline keyboard, with which he can get information about the game. If the game is not found, you must either make sure the name is spelled correctly (it must completely match the name of the game in the Google Play store, except for the case), or add the game to the library by sending the bot a link to the game. I deployed the bot on heroku and for those who in the future plan to write their own bot and host it for free on heroku, I will give a couple of recommendations for solving the difficulties that you may encounter (since I encountered them myself). Unfortunately, due to the nature of Heroku, the bot library is constantly “reset” once every 24 hours. My plan does not support storing files on Heroku servers, so it simply pulls my game file from Github. There were several solutions: use a database, or look for another server that would store this file with the game. I decided not to do anything for now, since essentially the bot is not that useful. I needed it rather to gain a full experience, which is basically what I achieved. So, recommendations for Heroku:

1. You will most likely have to register on heroku using a VPN if you live in Russia.

2. At the root of the project you need to put a file without an extension called Procfile. Its content should be like this: https://github.com/miroha/Telegram-Bot/blob/master/Procfile

3. In pom.xml, add the following lines according to the example , where in the mainClass tag indicate the path to the class that contains the main method: bot.BotApplication (if the BotApplication class is in the bot folder).

4. Do not build any project using mvn package commands, etc., Heroku will assemble everything for you.

5. It is advisable to add a gitignore to the project, for example this:

   
   # Log file
   *.log
   
   # Compiled resources
   target
   
   # Tests
   test
   
   # IDEA files
   .idea
   *.iml

6. Actually upload the project to github, and then connect the repository to Heroku (or use other methods, there are 3 of them, if I’m not mistaken).

7. If the download was successful ("Build succeeded"), be sure to go to Configure Dynos:

   and switch the slider, and then make sure that it is in the ON position (due to the fact that I did not do this, my bot did not work and I racked my brain for a couple of days and made a lot of unnecessary movements).

8. Hide the bot token on Github. To do this, you need to get the token from the environment variable:

   
   public class Bot extends TelegramLongPollingBot {
   
       private static final String BOT_TOKEN = System.getenv("TOKEN");
   
       @Override
       public String getBotToken() {
           return BOT_TOKEN;
       }
   ...
   }

   And then after deploying the bot, set this variable in the Heroku dashboard in the Settings tab (to the right of TOKEN there will be a VALUE field, copy your bot’s token there):

In total, in 2 months of working on my own project I:

• received a fully working project written in Java;
• learned to work with third-party API (Telegram Bot API);
• in practice, I delved deeper into serialization, worked a lot with JSON and the Jackson library (initially I used GSON, but there were problems with it);
• strengthened my skills when working with files, got acquainted with Java NIO;
• learned to work with configuration .xml files and accustomed myself to logging;
• improved proficiency in development environment (IDEA);
• learned to work with git and learned the value of gitignore;
• gained skills in web page parsing (Jsoup library);
• learned and used several design patterns;
• developed a sense and desire to improve code (refactoring);
• I learned to find solutions online and not to be shy about asking questions to which I couldn’t find an answer.

I don't know how useful or useless the bot turned out to be, or how pretty/ugly the code was, but the experience I got was definitely worth it. I felt a sense of responsibility for my project. Every now and then I want to improve it, add something new. When I was able to run it and see that everything worked the way I wanted, it was a real thrill. Isn't that the main thing? Enjoy what you do and enjoy every working line of code like the last chocolate bar. Therefore, if you are mastering programming, then my advice to you: do not stay here until level 40, but start your own project as early as possible. If anyone is interested, the source code of the project is here (rewritten for Spring): https://github.com/miroha/GooglePlayGames-TelegramBot For the last two months I have hardly been studying new material, since it seems to me that I have reached a dead end. Without work, I no longer see where to develop, except perhaps to teach the Spring Framework, which is what I plan to do in the next month. And then I’ll try to “rewrite” the bot using this framework. Ready to answer any questions. :) Good luck everyone! UPDATE from 07/07/2020 The repository with the bot in pure Java was lost (I deleted it, a copy remained on another local machine), but I downloaded the rewritten bot for Spring Boot: https://github.com/miroha/GooglePlayGames-TelegramBot