Back-end development specifications

This article describes the development, debugging, and maintenance requirements related to backend specifications as they apply to the project development and delivery process.

1. Create ModuleCode

ModuleCode: module code One module code corresponds to one controller, and it belongs to a unified speaking standard during the project development process.

Edit file: server/base/src/main/java/com/isxcode/zhiqingyun/base/constants/ModuleCode.java

• Requirement 1: Capitalize variable names
• Requirement 2: Parameters must be lowercase
• Requirement 3: Underline separation
• Requirement 4: Clear meaning

/**
 * Demo案例模块.
 */
String EXAMPLE_DEMO = "example_demo";

2. Create SQL script

Based on Flyway management Sql, currently only supports h2, MySQL and PostgreSQL databases

Edit file: spark-yun/backend/main/src/main/resouces/db/migration/h2/V30__GH-88.sql Edit file: spark-yun/backend/main/src/main/resouces/db/migration/MySQL/V30__GH-88.sql Edit file: spark-yun/backend/main/src/main/resouces/db/migration/postgres/V30__GH-88.sql

• Requirement 1: The file name V"Write the number in order"__"Github Issue number", V must be capitalized, and underscores must be two, for example, GH-88, it is written as V30__GH-88.sql
• Requirement 2: The table must contain system fields, such as create_by, create_date_time, last_modified_by, last_modified_date_time, version_number, deleted, tenant_id
• Requirement 3: SY_「xxx」 (SY is the abbreviation of spark_yun) There is no need to add S at the end of XXX. The table name should be all capitalized and separated by underscores.

create table if not exists SY_EXAMPLE_DEMO
(
    id                       varchar(200)  not null unique primary key comment '唯一id',
    demo_name                varchar(200)  not null comment '名称',
    demo_content             varchar(200)  null comment '内容',
    remark                   varchar(500)  null comment '备注',
    create_by                varchar(200)  not null comment '创建人',
    create_date_time         datetime      not null comment '创建时间',
    last_modified_by          varchar(200)  not null comment '更新人',
    last_modified_date_time   datetime      not null comment '更新时间',
    version_number           int           not null comment '版本号',
    deleted                  int default 0 not null comment '逻辑删除',
    tenant_id                varchar(200)  not null comment '租户id'
);

3. Create Entity

Implement object persistence based on Jpa, supporting Jpa optimistic locking, false deletion, multi-tenancy, and basic system fields Entities need to be mapped one-to-one with tables in the database

Edit file: spark-yun/backend/modules/src/main/java/com/isxcode/spark/modules/example/demo/entity/ExampleDemoEntity.java

• Requirement 1: The file must be created in the entity directory, modules/example/demo/entity
• Requirement 2: File name "XXX" Entity.java
• Requirement 3: The directory needs to be divided into multiple directories based on underscores, such as EXAMPLE_DEMO, which is under the example/demo/entity directory

@Data
@Entity
@SQLDelete(sql = "UPDATE SY_EXAMPLE_DEMO SET deleted = 1 WHERE id = ? and version_number = ?")
@Where(clause = "deleted = 0 ${TENANT_FILTER} ")
@Table(name = "SY_EXAMPLE_DEMO")
@JsonIgnoreProperties({"hibernateLazyInitializer"})
@EntityListeners(AuditingEntityListener.class)
public class ExampleDemoEntity {

    @Id
    @GeneratedValue(generator = "sy-id-generator")
    @GenericGenerator(name = "sy-id-generator", strategy = "com.isxcode.zhiqingyun.config.GeneratedValueConfig")
    private String id;

    private String demoName;
    
    private String demoContent;

    private String remark;

    @CreatedDate
    private LocalDateTime createDateTime;

    @LastModifiedDate
    private LocalDateTime lastModifiedDateTime;

    @CreatedBy
    private String createBy;

    @LastModifiedBy
    private String lastModifiedBy;

    @Version
    private Long versionNumber;

    @Transient
    private Integer deleted;

    private String tenantId;

    @PrePersist
    public void prePersist() {
        this.tenantId = ContextHolder.getTenantId();
    }
}

4. Create Repository

Interact with the database based on Jpa and add, delete, modify and check the database

Edit the file spark-yun/backend/modules/src/main/java/com/isxcode/spark/modules/example/demo/repository/ExampleDemoRepository.java

• Requirement 1: The file must be created in the repository directory, modules/example/demo/repository
• Requirement 2: File name "XXX" Repository.java
• Requirement 3: Do not join more than three tables

Function naming convention:

• findByXXX: Query a single object
• findXXXByXXX: Query a custom single object
• findAllByXXX: Query object array
• findAllXXXByXXX: Query custom object array
• pageByXXX: paging object query
• pageSearchByXXX: paging search object query
• pageSearchXXXByXXX: paging search custom object query
• deleteByXXX: delete object
• deleteAllByXXX: delete objects in batches
• save: save a single object
• saveAll: Save objects in batches
• updateXXXByXXX: update object, not recommended
• getXXXByXXX: Get a specific return single value
• countXXXByXXX: Get the total number

Common parameter naming conventions:

• searchKeyWord: search key field
• pageable: paging request object

@Repository
@CacheConfig(cacheNames = {ModuleCode.EXAMPLE_DEMO})
public interface ExampleDemoRepository extends JpaRepository<ExampleDemoEntity, String> {
    
    Optional<DemoIspongEntity> findByAgeAndUsername(Integer age, String username);

    void deleteByAgeAndUsername(Integer age, String username);

    List<DemoIspongEntity> findAllByAgeAndUsername(Integer age, String username);

    // 自定义更新，非特殊情况禁止使用
    @Modifying
    @Query(value = "update ExampleDemoEntity set content = :content where name = :name")
    void updateContentByName(@Param("content") String content, @Param("name") String name);

    // 对象分页搜索查询
    @Query("SELECT D FROM ExampleDemoEntity D WHERE D.name LIKE %:searchKeyWord% order by D.createDateTime desc")
    Page<DemoIspongEntity> searchAll(@Param("searchKeyWord") String searchKeyWord, Pageable pageable);
}

5. Create AO object

Multi-table association query returns object "XXX" AO.java, Association Object joint object

• Requirement 1: Note that you must write a constructor and cannot use lombok
• Requirement 2: jpa writing method, the order of database fields and parameters in the constructor should be consistent
• Requirement 3: It is best not to join more than three tables.
• Requirement 4: The file must be created in the ao directory, spark-yun/backend/api/src/main/java/com/isxcode/spark/api/example/demo/ao
• Requirement 5: File name "XXX" Ao.java

@Data
public class ExampleInfoAo {

  private String username;

  private String content;

  public ExampleInfoAo(String username,String content) {
    this.username = username;
    this.content = content;
  }
}

@Query(value =
  "select "
    + " new com.isxcode.zhiqingyun.api.instance.ao.WorkflowInstanceAo("
    + "   U.account,"
    + "   E.content) "
    + "from ExampleDemoEntity E left join UserEntity U on D.id = U.id "
    + "where U.username LIKE %:keyword% AND E.tenantId=:tenantId order by D.createDateTime desc")
Page<ExampleInfoAo> pageExampleInfoAo(
  @Param("tenantId") String tenantId,
  @Param("searchKeyWord") String searchKeyWord,
  Pageable pageable);

6. Create Mapper

Based on the MapStruct object conversion layer, A object converts B object

@Mapper(componentModel = "spring",unmappedTargetPolicy = ReportingPolicy.IGNORE)
public interface DemoIspongMappers {
      
      @Mapping(target = "node",expression = "java( engineEntity.getActiveNodeNum()+ \"/\" +engineEntity.getAllNodeNum())")
      PageClusterRes clusterEntityToPageClusterRes(ClusterEntity clusterEntity);
}

7. Create Service

General business layer, it is forbidden to reference other services

8. Create BizService

Business layer, one-to-one correspondence with controller

9. Create BizImplService

Business implementation layer, used to extend the business layer, can create multiple

10.Create Controller

External interface management

11. Create Req

Request body

12. Create Res

response body

13. Create Constant

static variable

14. Create Dto

Procedure Stored Object Data To Object

15.Modify Properties

backend/api

All java basic objects:

Pojos: business objects, req, res, bo, dto, vo Constants: static values Exceptions: exception class Properties: Spring configuration class

backend/modules

All module logic codes:

Controller: interface layer Mapper: object transformation layer Entity: database object layer Repository: data persistence layer Service: business service layer

vip/backend

Similar to modules structure, this code is closed source code

service call Note: When you need to query all tenant data, use allData to wrap the query

public Page<PageMetaTableRes> pageMetaTable(PageMetaTableReq pageMetaTableReq) {

    Page<MetaTableAo> metaTablePage = allData(() -> metaTableRepository.searchAll(
            ContextHolder.getTenantId(),
            pageMetaTableReq.getSearchKeyWord(), 
            pageMetaTableReq.getDatasourceId(), PageRequest.of(pageMetaTableReq.getPage(), pageMetaTableReq.getPageSize())));

    return metaTablePage.map(metaMapper::metaTableEntityToPageMetaTableRes);
}

Controller

Management interface information based on OpenApi (Swagger) com.isxcode.zhiqingyun.modules.<module_code>.controller.<module_code>Controller All interfaces must use Post Interface naming convention:

• Add object interface: addXXX
• Update object interface: updateXXX
• Conditional update interface: updateXXXByXXX
• Paging search query interface: pageSearchXXX
• Paging query interface: pageXXX
• Delete object interface: deleteXXX
• Delete batch object interface: deleteAllXXX
• Query a single object interface: findXXX
• Conditional query for a single object interface: findXXXByXXX
• Return array object interface: findAllXXX
• Conditional query returns array object interface: findAllXXXByXXX
• Other behavior interfaces: action/check/remove/setXXX
• Password-free interface: openXXX

Case:

@Tag(name = "计算集群模块")
@RestController
@RequestMapping(ModuleCode.CLUSTER)
@RequiredArgsConstructor
public class ClusterController {

  // 只可以调用BizService服务层
  private final ClusterBizService clusterBizService;

  // 各前缀需保持一致
  @Operation(summary = "添加计算引擎接口")
  @PostMapping("/addCluster")
  @SuccessResponse("添加成功")
  public AddClusterRes addCluster(@Valid @RequestBody AddClusterReq addClusterReq) {

    return clusterBizService.addCluster(addClusterReq);
  }
     
  // 注解控制权限
  @Secured({RoleType.TENANT_MEMBER, RoleType.TENANT_SUPER_ADMIN})
  @Operation(summary = "添加数据源接口")
  @PostMapping("/addDatasource")
  @SuccessResponse("添加成功")
  public void addDatasource(@Valid @RequestBody AddDatasourceReq addDatasourceReq) {

    datasourceBizService.addDatasource(addDatasourceReq);
  }
  
  // 免密接口
  @Operation(summary = "用户登录接口")
  @PostMapping("/open/openLogin")
  @SuccessResponse("登录成功")
  public OpenLoginRes openLogin(@Valid @RequestBody OpenLoginReq openLoginReq) {
  
    return userBizService.openLogin(openLogingReq);
  }
  
  // 分页查询接口
  @Operation(summary = "查询所有启用用户接口")
  @PostMapping("/pageEnableUser")
  @SuccessResponse("查询成功")
  public Page<PageEnableUserRes> pageEnableUser(@Valid @RequestBody PageEnableUserReq pageEnableUserReq) {

    return userBizService.pageEnableUser(pageEnableUserReq);
  }
  
  // 文件上传接口
  @Operation(summary = "作业流导入接口(Swagger有Bug不能使用)")
  @PostMapping("/importWorkflow")
  @SuccessResponse("导入成功")
  public void importWorkflow(
        @RequestParam("workflowConfigFile") MultipartFile workflowConfigFile,
        @Schema(description = "作业流唯一id", example = "sy_ba1f12b5c8154f999a02a5be2373a438") @RequestParam(required = false)
            String workflowId) {
    
     workflowBizService.importWorkflow(workflowConfigFile, workflowId);
  }
  
  // 文件下载接口，不需要使用@SuccessResponse
  @Operation(summary = "作业流导出接口")
  @PostMapping("/exportWorkflow")
  public void exportWorkflow(@Valid @RequestBody ExportWorkflowReq exportWorkflowReq, HttpServletResponse response) {
    
    workflowBizService.exportWorkflow(exportWorkflowReq, response);
  }
}

Service

The business service code is divided into two parts:

• XXXService: Universal service layer, which can be called by any Biz service layer, but is prohibited from calling any Biz service layer.
• XXXBizService: Business logic service layer, which maintains one-to-one correspondence with the interfaces in the Controller.

com.isxcode.zhiqingyun.modules.<module_code>.service.<module_code>Servicecom.isxcode.zhiqingyun.modules.<module_code>.service.biz.<module_code>BizService

For example: General service layer Service Strictly use Javadoc to write comments, which will be reused many times later.

@Service
@RequiredArgsConstructor
public class ClusterNodeService {
        
    /**
     * <p>
     *     获取代理默认端口号
     * </p>
     *
     * @param agentPort 代理端口号
     * @return 代理端口号
     */
    public String getDefaultAgentPort(String agentPort) {

      if (Strings.isEmpty(agentPort)) {
        return sparkYunProperties.getDefaultAgentPort();
      } else {
        return agentPort;
      }
    }   
}

For example: business logic service layer <module_code>BizService

@Service
@RequiredArgsConstructor
public class ClusterBizService {

    // 添加对象
    public void addCluster(AddClusterReq addClusterReq) {

      ClusterEntity cluster = clusterMapper.addEngineReqToClusterEntity(addClusterReq);
      clusterRepository.save(cluster);
    }
    
    // 更新对象
    public void updateCluster(UpdateClusterReq updateClusterReq) {
    
      ClusterEntity cluster = clusterService.getCluster(updateClusterReq.getClusterId());
      cluster = clusterMapper.updateEngineReqToClusterEntity(updateClusterReq, cluster);
      clusterRepository.save(cluster);
    }
    
    // 分页搜索查询对象
    public Page<PageClusterRes> pageCluster(PageClusterReq pageClusterReq) {
    
      Page<ClusterEntity> clusterPage = clusterRepository.pageCluster(
              pageClusterReq.getSearchKeyWord(),
              PageRequest.of(pageClusterReq.getPage(), pageClusterReq.getPageSize()));
       
      // 使用mapper转换对象
      return clusterPage.map(clusterMapper::clusterEntityToPageClusterRes);
    }
}

Pojos

Object type is prohibited Support Spring Validation Base objects:

• XXXReq: Request object (Request), corresponds to the interface one-to-one, and cannot be reused (including sub-object Dto)
• XXXRes: Response object (Response), which corresponds to the interface one-to-one and cannot be reused (including sub-object Vo)
• XXXVo: Front-end display data (View Object) in the response object, which can be reused
• XXXBo: The object (Business Object) of the interface in the general service layer Service, which can be reused
• XXXDto: Data Transfer Object, reusable
• XXXReq, paging request object

@EqualsAndHashCode(callSuper = true)
@Data
public class PageUserReq extends BasePageRequest {

}

XXXReq, field checksum comments are required

@Data
public class AddUserReq {
  
  @Schema(title = "名字", example = "仅供内部使用")
  @NotEmpty(message = "用户名称不能为空")
  @Size(min = 1, max = 100, message = "名称长度5～100")
  private String username;
  
  @Schema(title = "邮箱", example = "ispong@123.com")
  @Email(message = "邮箱不合法")
  private String email;
}

XXXRes, Null is not returned to the front end

@Data
@AllArgsConstructor
@NoArgsConstructor
@Builder
@JsonInclude(JsonInclude.Include.NON_NULL)
public class PageClusterRes {

  private String id;
  
}

Properties

Spring configuration class com.isxcode.zhiqingyun.api.main.properties.SparkYunProperties

@Data
@Component
@ConfigurationProperties(prefix = "spark-yun")
@EnableConfigurationProperties(SparkYunProperties.class)
public class SparkYunProperties {

  /** 可以让脚本临时复制的目录. */
  private String tmpDir = "/tmp";

  /** 代理默认端口号. */
  private String defaultAgentPort = "30177";
}

Exceptions

Exception class, IsxAppException is used by default

if (!tenantUserEntityOptional.isPresent()) {
  throw new IsxAppException("用户不存在");
}

Get user variables How to get the userId and tenantId of the current user

import com.isxcode.zhiqingyun.common.security.ContextHolder;

System.out.println(ContextHolder.getTenantId());
System.out.println(ContextHolder.getUserId());

Swagger specification

There is no need to write interface documents later, but swagger must be strictly written.

Req

package com.isxcode.zhiqingyun.api.cluster.pojos.req;

import io.swagger.v3.oas.annotations.media.Schema;
import javax.validation.constraints.NotEmpty;
import lombok.Data;

@Data
public class AddClusterReq {

  @Schema(title = "计算引擎名称", example = "至轻云内部计算引擎")
  @NotEmpty(message = "引擎名称不能为空")
  private String name;

  @Schema(title = "计算引擎类型", example = "
      写清楚，类型的枚举值
      Yarn
      K8s
      Spark
  ")
  @NotEmpty(message = "计算引擎类型不能为空")
  private String clusterType;

  @Schema(title = "备注", example = "仅供内部使用")
  private String remark;
}

Res

package com.isxcode.zhiqingyun.api.cluster.pojos.req;

import io.swagger.v3.oas.annotations.media.Schema;
import javax.validation.constraints.NotEmpty;
import lombok.Data;

@Data
public class AddClusterReq {

    // 标记 是否为后端逻辑字段，前端不用填充，说明清楚
  @Schema(title = "计算引擎名称", example = "至轻云内部计算引擎")
  @NotEmpty(message = "引擎名称不能为空")
  private String name;

  @Schema(title = "计算引擎类型", example = "
      写清楚，类型的枚举值
      Yarn
      K8s
      Spark
  ")
  @NotEmpty(message = "计算引擎类型不能为空")
  private String clusterType;

  @Schema(title = "备注", example = "仅供内部使用")
  private String remark;
}

Controller

package com.isxcode.zhiqingyun.modules.datasource.controller;

import com.isxcode.zhiqingyun.api.datasource.pojos.res.GetConnectLogRes;
import com.isxcode.zhiqingyun.api.datasource.pojos.res.PageDatasourceRes;
import com.isxcode.zhiqingyun.api.datasource.pojos.res.TestConnectRes;
import com.isxcode.zhiqingyun.base.constants.ModuleCode;
import com.isxcode.zhiqingyun.base.constants.RoleType;
import com.isxcode.zhiqingyun.common.annotations.successResponse.SuccessResponse;
import com.isxcode.zhiqingyun.modules.datasource.service.biz.DatasourceBizService;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;

import javax.validation.Valid;

import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page;
import org.springframework.security.access.annotation.Secured;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@Tag(name = "数据源模块")
@RestController
@RequestMapping(ModuleCode.DATASOURCE)
@RequiredArgsConstructor
public class DatasourceController {

    private final DatasourceBizService datasourceBizService;

    @Secured({RoleType.TENANT_SUPER_ADMIN})
    @Operation(summary = "添加数据源接口")
    @PostMapping("/addDatasource")
    @SuccessResponse("添加成功")
    public void addDatasource(@Valid @RequestBody AddDatasourceReq addDatasourceReq) {

        datasourceBizService.addDatasource(addDatasourceReq);
    }

    @Secured({RoleType.TENANT_SUPER_ADMIN})
    @Operation(summary = "更新数据源接口")
    @PostMapping("/updateDatasource")
    @SuccessResponse("更新成功")
    public void updateDatasource(@Valid @RequestBody UpdateDatasourceReq updateDatasourceReq) {

        datasourceBizService.updateDatasource(updateDatasourceReq);
    }

    @Secured({RoleType.TENANT_MEMBER, RoleType.TENANT_SUPER_ADMIN})
    @Operation(summary = "查询数据源列表接口")
    @PostMapping("/pageDatasource")
    @SuccessResponse("查询数据源成功")
    public Page<PageDatasourceRes> pageDatasource(@Valid @RequestBody PageDatasourceReq pageDatasourceReq) {

        return datasourceBizService.pageDatasource(pageDatasourceReq);
    }

    @Secured({RoleType.TENANT_SUPER_ADMIN})
    @Operation(summary = "删除数据源接口")
    @PostMapping("/deleteDatasource")
    @SuccessResponse("删除成功")
    public void deleteDatasource(@Valid @RequestBody DeleteDatasourceReq deleteDatasourceReq) {

        datasourceBizService.deleteDatasource(deleteDatasourceReq);
    }

    @Secured({RoleType.TENANT_MEMBER, RoleType.TENANT_SUPER_ADMIN})
    @Operation(summary = "测试数据源连接接口")
    @PostMapping("/testConnect")
    @SuccessResponse("检测完成")
    public TestConnectRes testConnect(@Valid @RequestBody GetConnectLogReq testConnectReq) {

        return datasourceBizService.testConnect(testConnectReq);
    }

    @Secured({RoleType.TENANT_MEMBER, RoleType.TENANT_SUPER_ADMIN})
    @Operation(summary = "查询连接日志")
    @PostMapping("/getConnectLog")
    @SuccessResponse("获取成功")
    public GetConnectLogRes getConnectLog(@Valid @RequestBody GetConnectLogReq getConnectLogReq) {

        return datasourceBizService.getConnectLog(getConnectLogReq);
    }
}

Database DO object verification

// 用上面的方法  getXXXEntity
WorkEntity work = workService.getWorkEntity(saveSyncWorkConfigReq.getWorkId());

// 不要重复写
Optional<WorkEntity> workEntityOptional = workRepository.findById(saveSyncWorkConfigReq.getWorkId());
if (!workEntityOptional.isPresent()) {
  throw new IsxAppException("作业不存在");
}

Multi-table related paging query

Construct multi-table return object MetaColumnAo

@Data
public class MetaColumnAo {

    private String datasourceId;

    private String tableName;

    private String columnName;

    private String columnComment;

    private String columnType;

    private String customComment;

    private LocalDateTime lastModifiedDateTime;

    public MetaColumnAo(String datasourceId, String tableName, String columnName, String columnComment,
        String columnType, String customComment, LocalDateTime lastModifiedDateTime) {

        this.columnType = columnType;
        this.datasourceId = datasourceId;
        this.tableName = tableName;
        this.columnName = columnName;
        this.columnComment = columnComment;
        this.customComment = customComment;
        this.lastModifiedDateTime = lastModifiedDateTime;
    }
}

Write multi-table query sql Note: Be sure to specify the tenantId field

@Repository
@CacheConfig(cacheNames = {ModuleCode.VIP_META})
public interface MetaColumnRepository extends JpaRepository<MetaColumnEntity, MetaColumnId> {
    
    @Query("SELECT " +
            "new com.isxcode.zhiqingyun.api.meta.ao.MetaColumnAo(" +
            "   M.datasourceId," +
            "   M.tableName," +
            "   M.columnName," +
            "   M.columnComment," +
            "   M.columnType," +
            "   MI.customComment," +
            "   M.lastModifiedDateTime) FROM MetaColumnEntity M " +
            "left join MetaColumnInfoEntity MI " +
            "on M.datasourceId=MI.datasourceId and M.tableName = MI.tableName and M.columnName=MI.columnName " +
            "WHERE M.tenantId=:tenantId " +
            "   AND " +
            "(M.columnName LIKE %:keyword% OR M.columnComment LIKE %:keyword% OR M.tableName LIKE %:keyword% OR MI.customComment LIKE %:keyword%) " +
            "order by M.createDateTime desc")
    Page<MetaColumnAo> searchAll(@Param("tenantId") String tenantId, @Param("keyword") String searchKeyWord,
        Pageable pageable);
}

Code specifications

• It is forbidden to comment code in the project, just delete the code directly
• Keyword naming is prohibited. Example: connect
• Do not comment the code, delete it directly!!! Commenting the code is very annoying to read
• Use - as separator for easy readability
• Static in java, use _ as separator
• The use of unusual abbreviations or self-defined abbreviations is prohibited
• Exceptions caught by the backend must be log.error(e.getMessage(),e)