Saving files to the application and data about them on the database

Let's imagine that you are working on your web application. In my articles, I consider individual pieces of this puzzle, such as:

• Database integration testing with MariaDB to replace MySql
• Implementation of multilingual application

Why are these themes useful? And the fact that these examples are very close to working on real projects, and probing these topics will be very useful for you. Today we will take the next piece of this puzzle - working with files, since in our time you can’t even meet a site that does not interact with them (for example, all sorts of web shops, social networks, and so on). The review will be carried out using the upload/download/delete methods as an example , we will save to a folder (in resource) in our application, so as not to complicate things. The idea is that we will save, in addition to the file itself in our so-called file storage (storage), an entity with information about our file (size, name, etc.) in our database - a previously created table. That is, when loading a file, this entity will be very useful to us, and when deleting, we should not forget about it in any way. Let's take a look at this table: And let's put AUTO_INCREMENT on id, as adults, to automatically generate an identifier at the database level. First, let's take a look at our structure: Entity under the table shown above:

@Builder(toBuilder = true)
@Getter
@ToString
public class FileInfo {

   private Long id;

   private String name;

   private Long size;

   private String key;

   private LocalDate uploadDate;
}

Upload

Let's look at the controller:

@RestController
@RequestMapping("/file")
@RequiredArgsConstructor
public class FileController {

   private final FileService fileService;

   @PostMapping
   public ResponseEntity<FileInfo> upload(@RequestParam MultipartFile attachment) {
       try {
           return new ResponseEntity<>(fileService.upload(attachment), HttpStatus.CREATED);
       } catch (IOException e) {
           return new ResponseEntity<>(HttpStatus.BAD_REQUEST);
       }
   }

From the interesting: 9 - We accept the file in the form of MultipartFile. It can also be received as an array of bytes, but I like this option more, since we can MultipartFilepull out various properties of the transferred file. 10 - 14 - we wrap our actions in try catchso that if an exception occurs at a lower level, we will forward it higher and send a 400 error as a response. Next is the level of service:

public interface FileService {

   FileInfo upload(MultipartFile resource) throws IOException;

Let's see the implementation:

@Service
@RequiredArgsConstructor
public class FileServiceImpl implements FileService {

   private final FileDAO fileDAO;
   private final FileManager fileManager;

   @Transactional(rollbackFor = {IOException.class})
   @Override
   public FileInfo upload(MultipartFile resource) throws IOException {
       String key = generateKey(resource.getName());
       FileInfo createdFile = FileInfo.builder()
               .name(resource.getOriginalFilename())
               .key(key)
               .size(resource.getSize())
               .build();
       createdFile = fileDAO.create(createdFile);
       fileManager.upload(resource.getBytes(), key);

       return createdFile;
   }

8 - in the event of an IOException falling, all our saves in the database will be rolled back. 11 - generate a key that will be unique for the file when it is saved (even if two files with the same name are saved, there will be no confusion). 12 - we build an entity for saving in the database. 17 - we drive the entity with info into the database. 18 - save the file with a hashed name. 20 - we return the created entity FileInfo, but with the generated id in the database (this will be discussed a little later) and the date of creation. Method for generating a key to a file:

private String generateKey(String name) {
   return DigestUtils.md5Hex(name + LocalDateTime.now().toString());
}

Here we hash the name + date of creation, which will provide us with uniqueness. Layer dao interface:

public interface FileDAO {

   FileInfo create(FileInfo file);

Its implementation:

@Repository
@RequiredArgsConstructor
public class FileDAOImpl implements FileDAO {

   private static final String CREATE_FILE = "INSERT INTO files_info(file_name, file_size, file_key, upload_date) VALUES (?, ?, ?, ?)";

   private final JdbcTemplate jdbcTemplate;

   @Override
   public FileInfo create(final FileInfo file) {
       LocalDate uploadDate = LocalDate.now();
       GeneratedKeyHolder keyHolder = new GeneratedKeyHolder();
       jdbcTemplate.update(x -> {
           PreparedStatement preparedStatement = x.prepareStatement(CREATE_FILE, Statement.RETURN_GENERATED_KEYS);
           preparedStatement.setString(1, file.getName());
           preparedStatement.setLong(2, file.getSize());
           preparedStatement.setString(3, file.getKey());
           preparedStatement.setDate(4, Date.valueOf(uploadDate));
           return preparedStatement;
       }, keyHolder);

       return file.toBuilder()
               .id(keyHolder.getKey().longValue())
               .uploadDate(uploadDate)
               .build();
   }

11 - create a date which we will save. 12 - 21 - we save the entity, but in a more complicated way, with the explicit creation of an object PreparedStatementso that you can pull out the generated id (it does not pull it out with a separate request, but in the form of response metadata). 22 - 26 - we complete our long-suffering essence and give it up (in fact, it does not complete it, but creates a new object, filling in the transferred fields and copying the rest from the original one). Let's see how our files will be saved in FileManager:

public void upload(byte[] resource, String keyName) throws IOException {
   Path path = Paths.get(DIRECTORY_PATH, keyName);
   Path file = Files.createFile(path);
   FileOutputStream stream = null;
   try {
       stream = new FileOutputStream(file.toString());
       stream.write(resource);
   } finally {
       stream.close();
   }
}

1 - we accept the file as an array of bytes and separately the name under which it will be saved (our generated key). 2 - 3 - create a path (and write the path plus our key in the path) and a file along it. 6 - 7 - we create a stream and write our bytes there (and we wrap all this stuff in try-finallyto be sure that the stream will close for sure). However, many of the methods can throw an IOException for us. In this case, thanks to the forwarding specified in the method header, we will throw it into the controller and give 400 status. Let's test the whole thing in Postman: As you can see, everything is fine, the response is 201, the response JSON came in the form of our persisted entity in the database, and if we look at our storage: DB: we will see that we have a new value. (=*;*=)

Download

controller:

@GetMapping(path = "/{id}", produces = MediaType.APPLICATION_OCTET_STREAM_VALUE)
public ResponseEntity<Resource> download(@PathVariable("id") Long id) {
   try {
       FileInfo foundFile = fileService.findById(id);
       Resource resource = fileService.download(foundFile.getKey());
       return ResponseEntity.ok()
               .header("Content-Disposition", "attachment; filename=" + foundFile.getName())
               .body(resource);
   } catch (IOException e) {
       return new ResponseEntity<>(HttpStatus.NOT_FOUND);
   }
}

We also wrap in try-catchand in case of IOException we send 404 response status (not found). 4 - we pull out the adjacent FileInfo entity from the database. 5 - by key from the entity we download the file 6-8 - we send the file back, while adding the file name to the header (again, obtained from the entity with information about the file). Let's take a deeper look. Service interface:

Resource download(String key) throws IOException;

FileInfo findById(Long fileId);

Implementation:

@Override
public Resource download(String key) throws IOException {
   return fileManager.download(key);
}

@Transactional(readOnly = true)
@Override
public FileInfo findById(Long fileId) {
   return fileDAO.findById(fileId);
}

There is nothing particularly interesting here: the method of searching for an entity by id and downloading a file, except maybe 46 - we mark that we have a transaction for reading. dao level:

FileInfo findById(Long fileId);

Implementation:

private static final String FIND_FILE_BY_ID = "SELECT id, file_name, file_size, file_key, upload_date FROM files_info WHERE id = ?";

@Override
public FileInfo findById(Long fileId) {
   return jdbcTemplate.queryForObject(FIND_FILE_BY_ID, rowMapper(), fileId);
}

private RowMapper<FileInfo> rowMapper() {
   return (rs, rowNum) -> FileInfo.builder()
           .id(rs.getLong("id"))
           .name(rs.getString("file_name"))
           .size(rs.getLong("file_size"))
           .key(rs.getString("file_key"))
           .uploadDate(rs.getObject("upload_date", LocalDate.class))
           .build();
}

4 — search by id using jdbcTemplateand RowMapper. 8 - 15 - implementation RowMapperfor our specific case, for matching data from the database and model fields. Let's go to FileManagerand see how our file is loaded:

public Resource download(String key) throws IOException {
   Path path = Paths.get(DIRECTORY_PATH + key);
   Resource resource = new UrlResource(path.toUri());
   if (resource.exists() || resource.isReadable()) {
       return resource;
   } else {
       throw new IOException();
   }
}

We return the file as an object Resource, and we will search by the key. 3 - create Resourcealong the way + key. 4 - 8 - check that the file at the given path is not empty and read it. If everything is OK, we return it, and if not, we throw the IOException up. Checking our method in Postman: As you can see, it worked OK))

Delete

@DeleteMapping(value = "/{id}")
public ResponseEntity<Void> delete(@PathVariable("id") Long id) {
   try {
       fileService.delete(id);
       return new ResponseEntity<>(HttpStatus.OK);
   } catch (IOException e) {
       return new ResponseEntity<>(HttpStatus.NOT_FOUND);
   }
}

Nothing special here: we also return 404 on failure with try-catch. Service interface:

void delete(Long fileId) throws IOException;

implementation:

@Transactional(rollbackFor = {IOException.class})
@Override
public void delete(Long fileId) throws IOException {
   FileInfo file = fileDAO.findById(fileId);
   fileDAO.delete(fileId);
   fileManager.delete(file.getKey());
}

1 - also rollback of data modification (deletion) when IOException is thrown. 5 - delete information about the file from the database. 6 - delete the file itself from our "storage". dao interface:

void delete(Long fileId);

Implementation:

private static final String DELETE_FILE_BY_ID = "DELETE FROM files_info WHERE id = ?";

@Override
public void delete(Long fileId) {
   jdbcTemplate.update(DELETE_FILE_BY_ID, fileId);
}

Nothing like that - just delete. Deleting the file itself:

public void delete(String key) throws IOException {
       Path path = Paths.get(DIRECTORY_PATH + key);
       Files.delete(path);
   }
}

We use Postman: We look in the storage: Empty :) Now in the database: We see that everything is good))

test

Let's try to write a test for our FileManager. First, let's take a look at the structure of the test part: mockFile.txt is the file with which we will test our file storage operations. testFileStorage will be our storage replacement. FileManagerTest:

public class FileManagerTest {

   private static MultipartFile multipartFile;

   private static FileManager manager;

   private static FileInfo file;

   @BeforeClass
   public static void prepareTestData() throws IOException {
       file = FileInfo.builder()
               .id(9L)
               .name("mockFile.txt")
               .key("mockFile.txt")
               .size(38975L)
               .uploadDate(LocalDate.now())
               .build();
       multipartFile = new MockMultipartFile("mockFile", "mockFile.txt", "txt",
               new FileInputStream("src/test/resources/mockFile.txt"));
       manager = new FileManager();
   }

Here we see the task of test data. File save test:

@Test
public void uploadTest() throws IOException {
   ReflectionTestUtils.setField(manager, "DIRECTORY_PATH", "src/test/resources/testFileStorage/");

   manager.upload(multipartFile.getBytes(), "mockFile.txt");

   Path checkFile = Paths.get("src/test/resources/testFileStorage/mockFile.txt");
   assertThat(Files.exists(checkFile)).isTrue();
   assertThat(Files.isRegularFile(checkFile)).isTrue();
   assertThat(Files.size(checkFile)).isEqualTo(multipartFile.getSize());
   Files.delete(checkFile);
}

3 - using test reflection, we change our constant in the service to set the path for saving the file. 5 - call the method being checked. 7 - 10 - check the correct execution of the save. 11 - delete the saved file (we should not leave any traces).

@Test
public void downloadTest() throws IOException {
   ReflectionTestUtils.setField(manager, "DIRECTORY_PATH", "src/test/resources/");

   Resource resource = manager.download(file.getKey());

   assertThat(resource.isFile()).isTrue();
   assertThat(resource.getFilename()).isEqualTo(file.getName());
   assertThat(resource.exists()).isTrue();
}

File upload test: 3 - again, change the path for our FileManager. 5 - we use the method being checked. 7 - 9 - check the result of execution. File deletion test:

@Test
public void deleteTest() throws IOException {
   Path checkFile = Paths.get("src/test/resources/testFileStorage/mockFile.txt");
   Files.createFile(checkFile);
   assertThat(Files.exists(checkFile)).isTrue();
   assertThat(Files.isRegularFile(checkFile)).isTrue();
   ReflectionTestUtils.setField(manager, "DIRECTORY_PATH", "src/test/resources/testFileStorage/");

   manager.delete(file.getKey());

   assertThat(Files.notExists(checkFile)).isTrue();
}

9 - 3 - 4 - set the path and create a file. 5 - 6 - check its existence. 9 - use the method being checked. 77 - check that the object is no longer there. And we look at what we have there according to dependencies:

<dependencies>
   <dependency>
       <groupId>org.springframework.boot</groupId>
       <artifactId>spring-boot-starter-data-jdbc</artifactId>
   </dependency>
   <dependency>
       <groupId>org.springframework.boot</groupId>
       <artifactId>spring-boot-starter-web</artifactId>
   </dependency>
   <dependency>
       <groupId>org.springframework.boot</groupId>
       <artifactId>spring-boot-starter-test</artifactId>
       <scope>test</scope>
       <exclusions>
           <exclusion>
               <groupId>org.junit.vintage</groupId>
               <artifactId>junit-vintage-engine</artifactId>
           </exclusion>
       </exclusions>
   </dependency>
   <dependency>
       <groupId>org.projectlombok</groupId>
       <artifactId>lombok</artifactId>
       <version>1.18.10</version>
       <scope>provided</scope>
   </dependency>
   <dependency>
       <groupId>mysql</groupId>
       <artifactId>mysql-connector-java</artifactId>
       <version>8.0.18</version>
   </dependency>
   <dependency>
       <groupId>commons-codec</groupId>
       <artifactId>commons-codec</artifactId>
       <version>1.9</version>
   </dependency>
   <dependency>
       <groupId>junit</groupId>
       <artifactId>junit</artifactId>
       <version>4.13-rc-2</version>
       <scope>test</scope>
   </dependency>
</dependencies>

That's all I have today))