Saving files to the application and data about them to the database

Let's imagine that you are working on your web application. In my articles I look at individual pieces of this mosaic, such as:

• Integration testing of a database using MariaDB to replace MySql
• Implementation of multilingual application

How are these topics useful? And the fact that these examples are very close to working on real projects, and testing these topics will be very useful for you. Today we will take the next piece of this mosaic - working with files, since nowadays you can no longer find a site that does not interact with them (for example, all sorts of web shops, social networks, and so on). The review will be carried out using the upload/download/delete methods as an example ; we will save it in a folder (in resource) in our application, so as not to complicate it. The idea is that we will save, in addition to the file itself in our so-called file storage, an entity with information about our file (size, name, etc.) in our database - a pre-created table. That is, when loading a file, this entity will be very useful to us, and when deleting, we must not forget about it in any way. Let's take a look at this table: And let's set the id to AUTO_INCREMENT like adults, to automatically generate an identifier at the database level. First, let's take a look at our structure: The entity under the table shown above:

@Builder(toBuilder = true)
@Getter
@ToString
public class FileInfo {

   private Long id;

   private String name;

   private Long size;

   private String key;

   private LocalDate uploadDate;
}

Upload

Let's look at the controller:

@RestController
@RequestMapping("/file")
@RequiredArgsConstructor
public class FileController {

   private final FileService fileService;

   @PostMapping
   public ResponseEntity<FileInfo> upload(@RequestParam MultipartFile attachment) {
       try {
           return new ResponseEntity<>(fileService.upload(attachment), HttpStatus.CREATED);
       } catch (IOException e) {
           return new ResponseEntity<>(HttpStatus.BAD_REQUEST);
       }
   }

Some interesting things: 9 - We accept the file in the form MultipartFile. It can also be received as an array of bytes, but I like this option better, since we can MultipartFileextract various properties of the transferred file. 10 - 14 - we wrap our actions in try catchso that if an exception occurs at a lower level, we forward it higher and send a 400 error as a response. Next is the service level:

public interface FileService {

   FileInfo upload(MultipartFile resource) throws IOException;

Let's look at the implementation:

@Service
@RequiredArgsConstructor
public class FileServiceImpl implements FileService {

   private final FileDAO fileDAO;
   private final FileManager fileManager;

   @Transactional(rollbackFor = {IOException.class})
   @Override
   public FileInfo upload(MultipartFile resource) throws IOException {
       String key = generateKey(resource.getName());
       FileInfo createdFile = FileInfo.builder()
               .name(resource.getOriginalFilename())
               .key(key)
               .size(resource.getSize())
               .build();
       createdFile = fileDAO.create(createdFile);
       fileManager.upload(resource.getBytes(), key);

       return createdFile;
   }

8 - in case of an IOException, all our saves in the database will be rolled back. 11 - we generate a key that will be unique for the file when it is saved (even if two files with the same names are saved, there will be no confusion). 12 — we build an entity to save in the database. 17 — we drive the entity with the information into the database. 18 - save the file with a hashed name. 20 — we return the created entity FileInfo, but with the generated id in the database (this will be discussed below) and the creation date. Method for generating a key to a file:

private String generateKey(String name) {
   return DigestUtils.md5Hex(name + LocalDateTime.now().toString());
}

Here we hash the name + creation date, which will ensure our uniqueness. Dao layer interface:

public interface FileDAO {

   FileInfo create(FileInfo file);

Its implementation:

@Repository
@RequiredArgsConstructor
public class FileDAOImpl implements FileDAO {

   private static final String CREATE_FILE = "INSERT INTO files_info(file_name, file_size, file_key, upload_date) VALUES (?, ?, ?, ?)";

   private final JdbcTemplate jdbcTemplate;

   @Override
   public FileInfo create(final FileInfo file) {
       LocalDate uploadDate = LocalDate.now();
       GeneratedKeyHolder keyHolder = new GeneratedKeyHolder();
       jdbcTemplate.update(x -> {
           PreparedStatement preparedStatement = x.prepareStatement(CREATE_FILE, Statement.RETURN_GENERATED_KEYS);
           preparedStatement.setString(1, file.getName());
           preparedStatement.setLong(2, file.getSize());
           preparedStatement.setString(3, file.getKey());
           preparedStatement.setDate(4, Date.valueOf(uploadDate));
           return preparedStatement;
       }, keyHolder);

       return file.toBuilder()
               .id(keyHolder.getKey().longValue())
               .uploadDate(uploadDate)
               .build();
   }

11 - create a date that we will save. 12 - 21 - we save the entity, but in a more complex way, with the explicit creation of the object PreparedStatementso that the generated id can be pulled out (it pulls it out not with a separate request, but in the form of response metadata). 22 - 26 - we complete the construction of our long-suffering entity and give it to the top (in fact, he does not complete it, but creates a new object, filling out the transferred fields and copying the rest from the original one). Let's see how our files will be saved in FileManager:

public void upload(byte[] resource, String keyName) throws IOException {
   Path path = Paths.get(DIRECTORY_PATH, keyName);
   Path file = Files.createFile(path);
   FileOutputStream stream = null;
   try {
       stream = new FileOutputStream(file.toString());
       stream.write(resource);
   } finally {
       stream.close();
   }
}

1 — we accept the file as an array of bytes and separately the name under which it will be saved (our generated key). 2 - 3 - create a path (and in the path we write the path plus our key) and a file along it. 6 - 7 - create a stream and write our bytes there (and wrap all this stuff in try-finallyto be sure that the stream will definitely close). However, many of the methods can throw IOException. In this case, thanks to the forwarding specified in the method header, we will pass it to the controller and give status 400. Let's test the whole thing in Postman: As you can see, everything is fine, the response is 201, the response JSON came in the form of our persisted entity in the database, and if we look into our storage: DB: we see that we have a new value. (=*;*=)

Download

Controller:

@GetMapping(path = "/{id}", produces = MediaType.APPLICATION_OCTET_STREAM_VALUE)
public ResponseEntity<Resource> download(@PathVariable("id") Long id) {
   try {
       FileInfo foundFile = fileService.findById(id);
       Resource resource = fileService.download(foundFile.getKey());
       return ResponseEntity.ok()
               .header("Content-Disposition", "attachment; filename=" + foundFile.getName())
               .body(resource);
   } catch (IOException e) {
       return new ResponseEntity<>(HttpStatus.NOT_FOUND);
   }
}

We also wrap it in try-catchand in case of IOException we send a 404 response status (not found). 4 - pull out the adjacent FileInfo entity from the database. 5 - using the key from the entity, download the file 6-8 - send back the file, adding the file name to the header (again, obtained from the entity with information about the file). Let's take a deeper look. Service interface:

Resource download(String key) throws IOException;

FileInfo findById(Long fileId);

Implementation:

@Override
public Resource download(String key) throws IOException {
   return fileManager.download(key);
}

@Transactional(readOnly = true)
@Override
public FileInfo findById(Long fileId) {
   return fileDAO.findById(fileId);
}

There is nothing particularly interesting here: the method of searching for an entity by id and downloading the file, except perhaps 46 - we mark that we have the transaction for reading. Dao level:

FileInfo findById(Long fileId);

Implementation:

private static final String FIND_FILE_BY_ID = "SELECT id, file_name, file_size, file_key, upload_date FROM files_info WHERE id = ?";

@Override
public FileInfo findById(Long fileId) {
   return jdbcTemplate.queryForObject(FIND_FILE_BY_ID, rowMapper(), fileId);
}

private RowMapper<FileInfo> rowMapper() {
   return (rs, rowNum) -> FileInfo.builder()
           .id(rs.getLong("id"))
           .name(rs.getString("file_name"))
           .size(rs.getLong("file_size"))
           .key(rs.getString("file_key"))
           .uploadDate(rs.getObject("upload_date", LocalDate.class))
           .build();
}

4 — search by id using jdbcTemplateand RowMapper. 8 - 15 - implementation RowMapperfor our specific case, for comparing data from the database and model fields. Let's go FileManagerand see how our file is loaded:

public Resource download(String key) throws IOException {
   Path path = Paths.get(DIRECTORY_PATH + key);
   Resource resource = new UrlResource(path.toUri());
   if (resource.exists() || resource.isReadable()) {
       return resource;
   } else {
       throw new IOException();
   }
}

We return the file as an object Resource, and we will search by key. 3 - create Resourcealong the path + key. 4 - 8 — we check that the file at the given path is not empty and read it. If everything is OK, we return it, and if not, we throw an IOException to the top. Let's check our method in Postman: As you can see, it worked OK))

Delete

@DeleteMapping(value = "/{id}")
public ResponseEntity<Void> delete(@PathVariable("id") Long id) {
   try {
       fileService.delete(id);
       return new ResponseEntity<>(HttpStatus.OK);
   } catch (IOException e) {
       return new ResponseEntity<>(HttpStatus.NOT_FOUND);
   }
}

Nothing special here: we also return 404 in case of failure using try-catch. Service interface:

void delete(Long fileId) throws IOException;

Implementation:

@Transactional(rollbackFor = {IOException.class})
@Override
public void delete(Long fileId) throws IOException {
   FileInfo file = fileDAO.findById(fileId);
   fileDAO.delete(fileId);
   fileManager.delete(file.getKey());
}

1 - also rollback of data changes (deletions) when an IOException occurs. 5 - delete information about the file from the database. 6 - delete the file itself from our “storage”. dao interface:

void delete(Long fileId);

Implementation:

private static final String DELETE_FILE_BY_ID = "DELETE FROM files_info WHERE id = ?";

@Override
public void delete(Long fileId) {
   jdbcTemplate.update(DELETE_FILE_BY_ID, fileId);
}

Nothing like that - just delete. Deleting the file itself:

public void delete(String key) throws IOException {
       Path path = Paths.get(DIRECTORY_PATH + key);
       Files.delete(path);
   }
}

We use Postman: We look in the storage: Empty :) Now in the database: We see that everything is good))

Test

Let's try to write a test for our FileManager. First, let's take a look at the structure of the test part: mockFile.txt is the file with which we will test our operations with file storage. testFileStorage will be a replacement for our storage. FileManagerTest:

public class FileManagerTest {

   private static MultipartFile multipartFile;

   private static FileManager manager;

   private static FileInfo file;

   @BeforeClass
   public static void prepareTestData() throws IOException {
       file = FileInfo.builder()
               .id(9L)
               .name("mockFile.txt")
               .key("mockFile.txt")
               .size(38975L)
               .uploadDate(LocalDate.now())
               .build();
       multipartFile = new MockMultipartFile("mockFile", "mockFile.txt", "txt",
               new FileInputStream("src/test/resources/mockFile.txt"));
       manager = new FileManager();
   }

Here we see the test data assignment. File saving test:

@Test
public void uploadTest() throws IOException {
   ReflectionTestUtils.setField(manager, "DIRECTORY_PATH", "src/test/resources/testFileStorage/");

   manager.upload(multipartFile.getBytes(), "mockFile.txt");

   Path checkFile = Paths.get("src/test/resources/testFileStorage/mockFile.txt");
   assertThat(Files.exists(checkFile)).isTrue();
   assertThat(Files.isRegularFile(checkFile)).isTrue();
   assertThat(Files.size(checkFile)).isEqualTo(multipartFile.getSize());
   Files.delete(checkFile);
}

3 — using test reflection, we change our constant in the service to set the path to save the file. 5 — call the method being tested. 7 - 10 — we check the correct execution of the save. 11 - delete the saved file (we should not leave any traces).

@Test
public void downloadTest() throws IOException {
   ReflectionTestUtils.setField(manager, "DIRECTORY_PATH", "src/test/resources/");

   Resource resource = manager.download(file.getKey());

   assertThat(resource.isFile()).isTrue();
   assertThat(resource.getFilename()).isEqualTo(file.getName());
   assertThat(resource.exists()).isTrue();
}

File upload test: 3 - again, change the path for our FileManager. 5 - use the method being tested. 7 - 9 — check the execution result. File deletion test:

@Test
public void deleteTest() throws IOException {
   Path checkFile = Paths.get("src/test/resources/testFileStorage/mockFile.txt");
   Files.createFile(checkFile);
   assertThat(Files.exists(checkFile)).isTrue();
   assertThat(Files.isRegularFile(checkFile)).isTrue();
   ReflectionTestUtils.setField(manager, "DIRECTORY_PATH", "src/test/resources/testFileStorage/");

   manager.delete(file.getKey());

   assertThat(Files.notExists(checkFile)).isTrue();
}

9 - 3 - 4 - set the path and create a file. 5 - 6 - we check its existence. 9 - we use a verifiable method. 77 - we check that the object is no longer there. And let’s see what we have in terms of dependencies:

<dependencies>
   <dependency>
       <groupId>org.springframework.boot</groupId>
       <artifactId>spring-boot-starter-data-jdbc</artifactId>
   </dependency>
   <dependency>
       <groupId>org.springframework.boot</groupId>
       <artifactId>spring-boot-starter-web</artifactId>
   </dependency>
   <dependency>
       <groupId>org.springframework.boot</groupId>
       <artifactId>spring-boot-starter-test</artifactId>
       <scope>test</scope>
       <exclusions>
           <exclusion>
               <groupId>org.junit.vintage</groupId>
               <artifactId>junit-vintage-engine</artifactId>
           </exclusion>
       </exclusions>
   </dependency>
   <dependency>
       <groupId>org.projectlombok</groupId>
       <artifactId>lombok</artifactId>
       <version>1.18.10</version>
       <scope>provided</scope>
   </dependency>
   <dependency>
       <groupId>mysql</groupId>
       <artifactId>mysql-connector-java</artifactId>
       <version>8.0.18</version>
   </dependency>
   <dependency>
       <groupId>commons-codec</groupId>
       <artifactId>commons-codec</artifactId>
       <version>1.9</version>
   </dependency>
   <dependency>
       <groupId>junit</groupId>
       <artifactId>junit</artifactId>
       <version>4.13-rc-2</version>
       <scope>test</scope>
   </dependency>
</dependencies>

That's all for me today))