Hello everyone, dear friends. In the last post we added admin commands, and today we'll talk about how to expand our statistics. Also, since the system is already essentially ready for MVP, we will do a little refactoring.
like - subscribe - bell put a star to our project , write comments and rate the article! For the penultimate time in this series of articles, I say see you soon!
Do you want to know immediately when a new project code is released? When is the new article coming out? Join my telegram channel . There I collect my articles, my thoughts, my open-source development together. |
Who is this article for?
The article is intended for those who have already read the entire series before. For those who are here for the first time - here is the beginning . Who can master this series? Yes, almost any person who understands Java Core. Everything else (as it seems to me) I give in a series of articles. For those who were waiting for the writing of the entire project in order to start figuring it out right away and not wait for new articles, you can already start, everything is practically done.Expanding statistics for the admin
This stage consists of two steps: planning and implementation. Let's get started.Planning updated bot statistics
Before taking on the functionality, it is worth considering what we want. What statistics would be interesting for the admin to track the work of the bot? At first, in order to find out the opinion of people, I created a post in the telegram channel . No proposals were received: the only suggestion was to leave everything as it is, because the idea is shown, but how to implement it is up to everyone to decide for themselves. I agree, and you need to have your own opinion, so I decided to highlight the data that are of interest to me:- you need to know the number of active users of the bot (this has already been done);
- you need to know the number of inactive users - let's add this. I think this is very useful, because it will be clear what is the proportion of users who refuse to use the bot among all users;
- the number of active users in each active group;
- average number of subscriptions per user.
Implementing the selected statistics
I don’t think that within the framework of this article we will really have time to implement everything, but I’m sure that we will need all of it. Based on the ideas above, we will make a DTO class with the fields that we want to receive:package com.github.codegymcommunity.jrtb.dto;
import lombok.Data;
import lombok.EqualsAndHashCode;
import java.util.List;
/**
* DTO for getting bot statistics.
*/
@Data
@EqualsAndHashCode
public class StatisticDTO {
private final int activeUserCount;
private final int inactiveUserCount;
private final List<GroupStatDTO> groupStatDTOs;
private final double averageGroupCountByUser;
}
and GroupStatDto
package com.github.codegymcommunity.jrtb.dto;
import lombok.Data;
import lombok.EqualsAndHashCode;
/**
* DTO for showing group id and title without data
*/
@Data
@EqualsAndHashCode(exclude = {"title", "activeUserCount"})
public class GroupStatDTO {
private final Integer id;
private final String title;
private final Integer activeUserCount;
}
We created it in the dto package next to service, bot, command and others. We will use these DTOs to pass data to the StatisticService (which we will create in a moment) and the StatCommand. Let's write a StatisticService:
package com.github.codegymcommunity.jrtb.service;
import com.github.codegymcommunity.jrtb.dto.StatisticDTO;
/**
* Service for getting bot statistics.
*/
public interface StatisticsService {
StatisticDTO countBotStatistic();
}
And its implementation:
package com.github.codegymcommunity.jrtb.service;
import com.github.codegymcommunity.jrtb.dto.GroupStatDTO;
import com.github.codegymcommunity.jrtb.dto.StatisticDTO;
import com.github.codegymcommunity.jrtb.repository.entity.TelegramUser;
import org.springframework.stereotype.Service;
import java.util.List;
import java.util.stream.Collectors;
import static org.springframework.util.CollectionUtils.isEmpty;
@Service
public class StatisticsServiceImpl implements StatisticsService {
private final GroupSubService groupSubService;
private final TelegramUserService telegramUserService;
public StatisticsServiceImpl(GroupSubService groupSubService, TelegramUserService telegramUserService) {
this.groupSubService = groupSubService;
this.telegramUserService = telegramUserService;
}
@Override
public StatisticDTO countBotStatistic() {
List<GroupStatDTO> groupStatDTOS = groupSubService.findAll().stream()
.filter(it -> !isEmpty(it.getUsers()))
.map(groupSub -> new GroupStatDTO(groupSub.getId(), groupSub.getTitle(), groupSub.getUsers().size()))
.collect(Collectors.toList());
List<TelegramUser> allInActiveUsers = telegramUserService.findAllInActiveUsers();
List<TelegramUser> allActiveUsers = telegramUserService.findAllActiveUsers();
double groupsPerUser = getGroupsPerUser(allActiveUsers);
return new StatisticDTO(allActiveUsers.size(), allInActiveUsers.size(), groupStatDTOS, groupsPerUser);
}
private double getGroupsPerUser(List<TelegramUser> allActiveUsers) {
return (double) allActiveUsers.stream().mapToInt(it -> it.getGroupSubs().size()).sum() / allActiveUsers.size();
}
}
Please note that following SOLID at the service level, we use only the services of other entities (GroupSubService, TelgeramUserService), and not their repositories. At first glance, this may look redundant, but it is not. This way we avoid problems with object dependencies with each other. TelegramUserService didn't have findAllInactiveUsers method , so let's create it in TelegramUserService:
/**
* Retrieve all inactive {@link TelegramUser}
*
* @return the collection of the inactive {@link TelegramUser} objects.
*/
List<TelegramUser> findAllInActiveUsers();
In TelegramUserServiceImple:
@Override
public List<TelegramUser> findAllInActiveUsers() {
return telegramUserRepository.findAllByActiveFalse();
}
The repository does not have such a method, so let's add it to the TelegramUserRepository:
List<TelegramUser> findAllByActiveFalse();
This is Spring Data, so we don't need to implement this method. The service was written, it's time to make a test for this case. We create StatisticServiceTest. Here we mock data from other services and check it, and based on the mocked data, we collect the correct GroupStatDTO object:
package com.github.codegymcommunity.jrtb.service;
import com.github.codegymcommunity.jrtb.dto.GroupStatDTO;
import com.github.codegymcommunity.jrtb.dto.StatisticDTO;
import com.github.codegymcommunity.jrtb.repository.entity.GroupSub;
import com.github.codegymcommunity.jrtb.repository.entity.TelegramUser;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.mockito.Mockito;
import static java.util.Collections.singletonList;
@DisplayName("Unit-level testing for StatisticsService")
class StatisticsServiceTest {
private GroupSubService groupSubService;
private TelegramUserService telegramUserService;
private StatisticsService statisticsService;
@BeforeEach
public void init() {
groupSubService = Mockito.mock(GroupSubService.class);
telegramUserService = Mockito.mock(TelegramUserService.class);
statisticsService = new StatisticsServiceImpl(groupSubService, telegramUserService);
}
@Test
public void shouldProperlySendStatDTO() {
//given
Mockito.when(telegramUserService.findAllInActiveUsers()).thenReturn(singletonList(new TelegramUser()));
TelegramUser activeUser = new TelegramUser();
activeUser.setGroupSubs(singletonList(new GroupSub()));
Mockito.when(telegramUserService.findAllActiveUsers()).thenReturn(singletonList(activeUser));
GroupSub groupSub = new GroupSub();
groupSub.setTitle("group");
groupSub.setId(1);
groupSub.setUsers(singletonList(new TelegramUser()));
Mockito.when(groupSubService.findAll()).thenReturn(singletonList(groupSub));
//when
StatisticDTO statisticDTO = statisticsService.countBotStatistic();
//then
Assertions.assertNotNull(statisticDTO);
Assertions.assertEquals(1, statisticDTO.getActiveUserCount());
Assertions.assertEquals(1, statisticDTO.getInactiveUserCount());
Assertions.assertEquals(1.0, statisticDTO.getAverageGroupCountByUser());
Assertions.assertEquals(singletonList(new GroupStatDTO(groupSub.getId(), groupSub.getTitle(), groupSub.getUsers().size())),
statisticDTO.getGroupStatDTOs());
}
}
Next, we need to update our StatCommand:
package com.github.codegymcommunity.jrtb.command;
import com.github.codegymcommunity.jrtb.command.annotation.AdminCommand;
import com.github.codegymcommunity.jrtb.dto.StatisticDTO;
import com.github.codegymcommunity.jrtb.service.SendBotMessageService;
import com.github.codegymcommunity.jrtb.service.StatisticsService;
import com.github.codegymcommunity.jrtb.service.TelegramUserService;
import org.springframework.beans.factory.annotation.Autowired;
import org.telegram.telegrambots.meta.api.objects.Update;
import java.util.stream.Collectors;
/**
* Statistics {@link Command}.
*/
@AdminCommand
public class StatCommand implements Command {
private final StatisticsService statisticsService;
private final SendBotMessageService sendBotMessageService;
public final static String STAT_MESSAGE = "✨<b>Подготовил статистику</b>✨\n" +
"- Количество активных пользователей: %s\n" +
"- Количество неактивных пользователей: %s\n" +
"- Среднее количество групп на одного пользователя: %s\n\n" +
"<b>Информация по активным группам</b>:\n" +
"%s";
@Autowired
public StatCommand(SendBotMessageService sendBotMessageService, StatisticsService statisticsService) {
this.sendBotMessageService = sendBotMessageService;
this.statisticsService = statisticsService;
}
@Override
public void execute(Update update) {
StatisticDTO statisticDTO = statisticsService.countBotStatistic();
String collectedGroups = statisticDTO.getGroupStatDTOs().stream()
.map(it -> String.format("%s (id = %s) - %s подписчиков", it.getTitle(), it.getId(), it.getActiveUserCount()))
.collect(Collectors.joining("\n"));
sendBotMessageService.sendMessage(update.getMessage().getChatId().toString(), String.format(STAT_MESSAGE,
statisticDTO.getActiveUserCount(),
statisticDTO.getInactiveUserCount(),
statisticDTO.getAverageGroupCountByUser(),
collectedGroups));
}
}
Here we are simply compiling all the information from the GroupStatDTO into a message for the admin. Since now we have StatCommand - not just a command, we need to write a separate test for it. Rewrite StatCommandTest:
package com.github.codegymcommunity.jrtb.command;
import com.github.codegymcommunity.jrtb.dto.GroupStatDTO;
import com.github.codegymcommunity.jrtb.dto.StatisticDTO;
import com.github.codegymcommunity.jrtb.service.SendBotMessageService;
import com.github.codegymcommunity.jrtb.service.StatisticsService;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.Mockito;
import java.util.Collections;
import static com.github.codegymcommunity.jrtb.command.AbstractCommandTest.prepareUpdate;
import static com.github.codegymcommunity.jrtb.command.StatCommand.STAT_MESSAGE;
import static java.lang.String.format;
@DisplayName("Unit-level testing for StatCommand")
public class StatCommandTest {
private SendBotMessageService sendBotMessageService;
private StatisticsService statisticsService;
private Command statCommand;
@BeforeEach
public void init() {
sendBotMessageService = Mockito.mock(SendBotMessageService.class);
statisticsService = Mockito.mock(StatisticsService.class);
statCommand = new StatCommand(sendBotMessageService, statisticsService);
}
@Test
public void shouldProperlySendMessage() {
//given
Long chatId = 1234567L;
GroupStatDTO groupDto = new GroupStatDTO(1, "group", 1);
StatisticDTO statisticDTO = new StatisticDTO(1, 1, Collections.singletonList(groupDto), 2.5);
Mockito.when(statisticsService.countBotStatistic())
.thenReturn(statisticDTO);
//when
statCommand.execute(prepareUpdate(chatId, CommandName.STAT.getCommandName()));
//then
Mockito.verify(sendBotMessageService).sendMessage(chatId.toString(), format(STAT_MESSAGE,
statisticDTO.getActiveUserCount(),
statisticDTO.getInactiveUserCount(),
statisticDTO.getAverageGroupCountByUser(),
format("%s (id = %s) - %s подписчиков", groupDto.getTitle(), groupDto.getId(), groupDto.getActiveUserCount())));
}
}
Here we have checked that exactly the message that we expect is being transmitted. Of course, the CommandContainer and CodeGymTelegramBot will need to be updated so that CommandStat now passes a StatisticCommand. I'll leave it up to you. But what will it look like? We run it on a test bot and write /stat, we get: Of course, this is now visible only to admins. Now it will be clearer to them what is happening with the bot.
In conclusion
Put the new SNAPSHOT version in pom.xml for the last time:<version>0.8.0-SNAPSHOT</version>
Since the version has been updated, it means that RELEASE_NOTES needs to be updated as well:
# Release Notes ## 0.8.0-SNAPSHOT * JRTB-10: extended bot statistics for admins.
It would seem, why fill it all out? Why write a description, who will read it? I'll tell you that just writing code is only a third (or even a quarter) of the job. Because someone should then read, expand, support this code. And documentation makes it possible to do it faster and better. All code changes can be found in this pull request . What do we have left? Only the last article with minor refactorings along with a retrospective. Let's play around before the release and analyze what we have come to in these 8 months.
Thoughts on the future of the bot
While I was preparing dinner, I was thinking about the future of the telegram bot, what else was left, what I wanted to do. And I realized that I didn’t touch on the topic of saving the state of the database (backup) of the data at all, so that I could restore it in case of something. I think that's how I would like to see it? So, to automate this process as much as possible. And against the background of these thoughts, I came to the following conclusion: I want database backups to be created from time to time and stored somewhere without my participation. But where to store? Definitely, this should be done outside the docker where the base is deployed. On the main server where the docker with the application is deployed, I also don’t really want to, because something can happen to the server and that’s all, by the way. And as a result, I came up with an idea. I must say right away that I don’t know if it can be implemented or not, but I like it the most. The idea itself:- Backups every day (week, month or other period of time) will be backed up to a special chat in a telegram, which admins will have access to, for example. Such a distributed data warehouse))
- Add an admin snapshot command for admin needs.
- Add a SUPER ADMIN command that would be able to restore the database using the provided file.
GO TO FULL VERSION