微服务common模块获取文件上传地址微服务文件上传下载

转载

墨舞天涯 2024-03-20 17:25:13

文章标签 微服务common模块获取文件上传地址 docker fastdfs System nginx 文章分类 运维

找到端口所在进程端口：netstat -ano| findstr 10001

微服务common模块获取文件上传地址微服务文件上传下载_nginx

删除进程： taskkill /f /pid 16340

把原来占用的进程删掉，使用docker部署
netstat -tunlp | grep 22122
tcp        0      0 0.0.0.0:22122           0.0.0.0:*               LISTEN      114901/fdfs_tracker 
 kill -9 114901

接下来进入正题：

安装FastDFS

微服务common模块获取文件上传地址微服务文件上传下载_nginx_02

结果：

微服务common模块获取文件上传地址微服务文件上传下载_微服务common模块获取文件上传地址_03

配置nginx

由于docker容器中已经集成了Nginx，我们只需要修改docker中的nginx配置，进入storage的容器内部：

docker exec -it storage /bin/bash

找到nginx(无需配置均可):cd etc/nginx,里面的配置文件nginx.conf文件：

注意nginx的监听端口是8080

微服务common模块获取文件上传地址微服务文件上传下载_fastdfs_04

文件上传微服务

1.FastDFS文件上传微服务配置客户端配置文件：fdfs_client.conf

#fastDFS的客户端访问配置文件
connect_timeout=60
network_timeout=60
charset=UTF-8
#Tracker的Http请求端口
http.tracker_http_port=8080
tracker_server=101.200.240.33:22122

2.application.yml

spring:
  application:
    name: topgame-service-file
  servlet:
    multipart:
      max-file-size: 10MB
      max-request-size: 10MB
server:
  port: 10004
eureka:
  client:
    service-url:
      defaultZone: http://127.0.0.1:7001/eureka/
  instance:
    prefer-ip-address: true
feign:
  hystrix:
    enabled: true

3.pom.xml

<dependencies>
        <dependency>
            <groupId>net.oschina.zcx7878</groupId>
            <artifactId>fastdfs-client-java</artifactId>
            <version>1.27.0.0</version>
        </dependency>
        <dependency>
            <groupId>com.topgame</groupId>
            <artifactId>topgame-common</artifactId>
            <version>1.0-SNAPSHOT</version>
        </dependency>
    </dependencies>

4.启动类

@EnableEurekaClient
@SpringBootApplication(exclude = DataSourceAutoConfiguration.class)
// 需要关闭数据库的自动配置
public class FileMain1004 {
    public static void main(String[] args) {
        SpringApplication.run(FileMain1004.class,args);
    }
}

5.上传所需工具类

1.FastDFSFile : FastDFS文件工具类，用于封装上传的文件

**
 * @author liang
 * @version 1.0
 * @date 2020/4/22 10:49
 * 封装文件上传信息
 * 时间
 * author
 * type
 * content
 * ext
 * md5
 * 附加信息
 */
@Data
@AllArgsConstructor
@NoArgsConstructor
public class FastDFSFile implements Serializable {
    private String name ;// 文件名
    private byte[] content;// 内容：字节数组
    private String ext ;// 扩展名:jpg , gif ....没有.
    private String md5 ;// 文件MD5摘要值
    private String author;// 文件创建作者
}

2.实现FastDFS的工具类：上传、下载等操作

/**
 * @author liang
 * @version 1.0
 * @date 2020/4/22 10:54
 * 实现FastDFS文件管理
 * 文件上传
 * 删除
 * 下载
 * 文件信息获取
 * storage信息获取
 * tracker信息获取
 */
public class FastDFSUtil {
    /**
     * 加载tracker连接信息
     */
    static {
        try {
            // 加载类路径下的文件
            ClassPathResource resource = new ClassPathResource("fdfs_client.conf");
            ClientGlobal.init(resource.getPath());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    /**
     * 文件上传
     * @param fastDFSFile : 封装的文件上传对象
     */
    public static String[] upload(FastDFSFile fastDFSFile) throws Exception {
        try {
            //附加参数
            NameValuePair[] metaList = new NameValuePair[1];
            metaList[0] = new NameValuePair("author",fastDFSFile.getAuthor());
            System.out.println("author:"+metaList[0].getValue()); //author:lxw成功获取
            //访问track1er:需要客户端
//        TrackerClient trackerClient = new TrackerClient();
//        //获取Tracker server连接信息
//        TrackerServer trackerServer = trackerClient.getConnection();//NullPointerException
            TrackerServer trackerServer = getTrackerServer();
//        //通过tracker获取storage
//        StorageClient storageClient = new StorageClient(trackerServer,null);
            StorageClient storageClient = getStorageClient(trackerServer);
            //Storage连接信息：创建出对象
            //操作：上传
            return storageClient.upload_file(fastDFSFile.getContent(),fastDFSFile.getExt(),metaList);
        }catch (Exception e){
            e.printStackTrace();
            return  null ; // 上传失败
        }
    }

    /**
     * 获取文件信息
     * @param groupName ：文件的组名--group1
     * @param remoteFileName : 文件的存储路径的名名字
     */
    public static FileInfo getFileInfo(String groupName,String remoteFileName)throws Exception{
        try {
            //创建TrackerClient,建议卸写在公共方法中
            TrackerServer trackerServer = getTrackerServer();
            StorageClient storageClient = getStorageClient(trackerServer);
            return storageClient.get_file_info(groupName, remoteFileName);
        }catch (Exception e){
            e.printStackTrace();
            return  null ;
        }
    }

    /**
     * 文件下载
     * @return : 字节输入流
     * @throws Exception
     */
    public static InputStream download(String groupName,String remoteFileName)throws Exception{
        System.out.println(groupName+","+remoteFileName);
        try {
            TrackerServer trackerServer = getTrackerServer();
            StorageClient storageClient = new StorageClient(trackerServer, null);
            //String group_name, String remote_filename :下载参数
            byte[] buffer = storageClient.download_file(groupName, remoteFileName);
            return new ByteArrayInputStream(buffer);//字节转输入流
        }catch (Exception e){
            e.printStackTrace();
            return  null ;
        }
    }

    /**
     * 根据文件的url找出组名：group1
     * @param url
     * @return
     */
    @Deprecated
    public static String getGroupNameByUrl(String url){
        String groupName = "" ;
        for (int i = 0 ; i < 2; i++){
            url = url.substring(url.indexOf("/")+1);
        }
        // 剩下：group1/M00/00/00/rBEeK17yq3SAfeLoAAI2WkTnmt4230.jpg",最后一部分是错误的
        groupName = url.substring(url.indexOf("/")+1);
        groupName = groupName.substring(0,groupName.indexOf("/"));
        return groupName ;
    }

    /**
     * 根据url获取文件名: M00/00/dASDASDA.jpg
     * @param url
     * @return
     */
    @Deprecated
    public static String getRemoteFileNameByUrl(String url){
        String origin = url ;
        String groupName = getGroupNameByUrl(url);
        return url.substring(origin.indexOf(groupName)+1+groupName.length());
    }

    /**
     * 删除文件
     * 注意清除nginx的缓存:add_header Cache-control no-store
     * 记载nginx.conf中的 location ~ /M00 中,再重启storage
     */
    public static boolean deleteFile(String groupName,String remoteFileName) throws Exception {
        TrackerServer trackerServer = getTrackerServer();
        StorageClient storageClient = new StorageClient(trackerServer, null);
        int i = storageClient.delete_file(groupName, remoteFileName);
        System.out.println(i);//2,删除成功返回的0,失败返回2
        return i == 0;
    }
    /**
     * 获取Storage信息
     */
    public static StorageServer getStorageServer() throws IOException {
        TrackerClient trackerClient = new TrackerClient();
        TrackerServer trackerServer = trackerClient.getConnection();
        return trackerClient.getStoreStorage(trackerServer);
    }
    /**
     * 获取Storage的IP和端口信息
     */
    public static ServerInfo[] getServerInfo(String groupName,String remoteFileName) throws IOException {
        TrackerClient trackerClient = new TrackerClient();
        TrackerServer trackerServer = trackerClient.getConnection();
        return trackerClient.getFetchStorages(trackerServer, groupName, remoteFileName);
    }
    /**
     * 获取Tracker信息
     */
    public static String getTackerInfo()throws Exception{
        TrackerServer trackerServer = getTrackerServer();
        int port = ClientGlobal.getG_tracker_http_port();// 配置文件信息:端口
        String ip = trackerServer.getInetSocketAddress().getHostString();
        return "http://"+ip+":"+port ;
    }

    /**
     * 代码优化1:获取trackerServer
     * @return
     * @throws Exception
     */
    public static TrackerServer getTrackerServer()throws Exception{
        TrackerClient trackerClient = new TrackerClient();
        TrackerServer trackerServer = trackerClient.getConnection();
        return trackerServer ;
    }
    /**
     * 代码优化2:获取StorageClient,方便直接操作文件
     * @return
     */
    private static StorageClient getStorageClient(TrackerServer trackerServer) throws IOException {
        //通过tracker获取storage
        return new StorageClient(trackerServer,null);
    }
    public static void main(String[] args) throws Exception {
        //写错了group名也行,但是其他的上传下载不行
//        FileInfo info = getFileInfo("xxx", "M00/00/00/rBEeK16f8QaAMbrLAABnrjUmJ9I081.jpg");
//        //最前面不要加上/
//        System.out.println(info);
        //得到的是内网ip,不是公网的
        //source_ip_addr = 172.17.30.43, file_size = 144986, create_timestamp = 2020-04-22 12:53:31, crc32 = 1156029150

        /*=======================文件下载测试=======================*/
        //将文件写入D:盘
//        InputStream is = null;
//        FileOutputStream fos = null ;
//         is = download("group1", "M00/00/00/rBEeK16fzcuADk8PAAI2WkTnmt4362.jpg");
//        fos = new FileOutputStream("D://1.jpg");
//        byte[] bytes = new byte[1024];
//        while (is.read(bytes) != -1){
//            fos.write(bytes);
//        }
//        fos.flush();
//        fos.close();
//        is.close();

        /*========================删除测试===============================*/
//        boolean flag = deleteFile("group1", "M00/00/00/rBEeK16f8QaAMbrLAABnrjUmJ9I081.jpg");
//        System.out.println(flag);
        /*==================获取storage信息======================*/
//        StorageServer storageServer = getStorageServer();
//        System.out.println(storageServer.getStorePathIndex());//存储路径索引
        /*=======================Storage组信息=====================*/
//        ServerInfo[] group1s = getServerInfo("group1", "M00/00/00/rBEeK16fzZaANHeOAAO8ZFZwros117.jpg");
//        for (ServerInfo group1 : group1s) {
//            System.out.println(group1.getPort()+":"+group1.getIpAddr());
//        }
        /*====================tracker信息=======================*/
        // System.out.println(getTackerInfo());
        System.out.println("monkeykeykey".indexOf("key"));// 3
        System.out.println("monkeykeykey".lastIndexOf("key"));// 9
    }

6.controller示例

package com.topgame.file.controller;

import cn.hutool.core.img.ImgUtil;
import com.github.pagehelper.PageInfo;
import com.topgame.entity.Result;
import com.topgame.entity.StatusCode;
import com.topgame.file.service.HeadPicService;
import com.topgame.file.util.FastDFSFile;
import com.topgame.file.util.FastDFSUtil;
import com.topgame.user.pojo.HeadPic;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.util.StringUtils;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.multipart.MultipartHttpServletRequest;

import javax.servlet.http.HttpServletRequest;
import javax.swing.*;
import java.io.InputStream;

/**
 * @author liang
 * @version 1.0
 * @date 2020/6/21 23:09
 * 用户头像上传微服务
 */
@RestController
@RequestMapping("headpic")
@CrossOrigin
public class HeadPicController {
    @Autowired
    private HeadPicService headPicService ;
    /**
     * 文件上传
     * @param headPic : 用户名
     * @param file : 图片文件
     */
    @PostMapping(value = "upload")
    public Result upload(@RequestParam(value = "file") MultipartFile file, HeadPic headPic, HttpServletRequest request)  {
//        MultipartHttpServletRequest multipartRequest = (MultipartHttpServletRequest) request;
//        MultipartFile multipartFile = multipartRequest.getFile("imageFile");
        String originalFilename = file.getOriginalFilename();
        String ext = originalFilename.substring(originalFilename.lastIndexOf(".")+1);
        if (!isImage(ext)){
            return new Result(false,StatusCode.ERROR,"您上传的文件不是图片类型,请重新上传！");
        }
        long size = file.getSize();
        System.out.println("size:"+size);// 5526 2084
        System.out.println(size > 10485760);// true
        if(size > 10485760){
            return new Result(false,StatusCode.ERROR,"文件大小超过10MB，上传失败！");
        }
        System.out.println(file+","+headPic);
        Result result = uploadUtil(file, headPic.getUsername());
        if (result.isFlag()){// 上传成功
            headPic = headPicService.add(headPic,file, (String) result.getData());
            return headPic == null ? result:new Result(true,StatusCode.OK,"上传成功!",headPic) ;
        }
        return result ;
    }

    /**
     * 判断是否为图片
     * @param ext ： 文件后缀名
     * @return
     */
    private boolean isImage(String ext){
        if (ext.equalsIgnoreCase(ImgUtil.IMAGE_TYPE_BMP)||ext.equalsIgnoreCase(ImgUtil.IMAGE_TYPE_GIF)||
        ext.equalsIgnoreCase(ImgUtil.IMAGE_TYPE_JPEG)|| ext.equalsIgnoreCase(ImgUtil.IMAGE_TYPE_JPG)||
        ext.equalsIgnoreCase(ImgUtil.IMAGE_TYPE_PNG)||ext.equalsIgnoreCase(ImgUtil.IMAGE_TYPE_PSD)){
            return true ;
        }
        return false ;
    }
    // 文件上传,FastDFSFile和用户名
    public Result uploadUtil(MultipartFile file,String username)  {
        FastDFSFile fastDFSFile = new FastDFSFile();
        fastDFSFile.setName(file.getOriginalFilename());
        String url="" ;//上传成功后的访问路径
        fastDFSFile.setAuthor(username);
        try {
            fastDFSFile.setContent(file.getBytes());
            fastDFSFile.setExt(StringUtils.getFilenameExtension(file.getOriginalFilename()));
            String[] uploads = FastDFSUtil.upload(fastDFSFile); // 为什么是null?
            /*成功的案例路径：http://topgamelxw.top:8080/group1/M00/00/00/ASJDKLJ4X.jpg
             * [0]:group1
             * [1]:后面剩下的
             * */
            url = FastDFSUtil.getTackerInfo()+"/"+uploads[0]+"/"+uploads[1];
            // 调用头像文件
            return new Result(true, StatusCode.OK,"文件上传成功",url);//访问成功
        } catch (Exception e) {
            e.printStackTrace();
        }
        return new Result(false, StatusCode.OK,"文件上传失败");
    }

    /**
     * 根据用户名查询出历史使用过的图片：按照上传时间排序并分页
     */
    @GetMapping("/{username}/{page}")
    public Result<PageInfo<HeadPic>> findHistory(@PathVariable("username") String username,@PathVariable("page") Integer page){
        PageInfo<HeadPic> headPicPageInfo = headPicService.findByUsername(username, page);
        return new Result<>(true,StatusCode.OK,"查询成功！",headPicPageInfo);
    }
    // 根据用户名查询出当前用使用的头像
    @GetMapping("{username}")
    public Result<HeadPic> getCurrHeadPic(@PathVariable("username") String username){
        return headPicService.findCurrHeadPicByUsername(username) ;
    }
    // 更新用户当前使用的
    @PutMapping("{username}")
    public Result<HeadPic> updateUsed(@PathVariable("username")String username, String path){
        System.out.println(username+path);
        return headPicService.updateUsed(username,path) ;
    }
    /**
     * 头像图片下载 ： 根据url进行下载
     * @return
     * 下载未完成
     */
    @GetMapping("download")
    public Result download(String url){
        try {
            InputStream download = FastDFSUtil.download(FastDFSUtil.getGroupNameByUrl(url), FastDFSUtil.getRemoteFileNameByUrl(url));
            return new Result<>(true,StatusCode.OK,"可以进行下载！",download);
        } catch (Exception e) {
            e.printStackTrace();
            return new Result<>(true,StatusCode.OK,"可以进行下载！");
//            return null ;
        }

    }

    public static void main(String[] args) {
        System.out.println(55262084L>10485760L);
    }
}

以上的HeadPic类：

package com.topgame.user.pojo;

import lombok.Data;
import org.hibernate.validator.constraints.br.CPF;

import javax.persistence.*;
import java.io.Serializable;
import java.util.Date;

/**
 * @author liang
 * @version 1.0
 * @date 2020/6/21 22:55
 * 用户头像实体类
 */
@Table(name = "tb_headpic")
@Data
public class HeadPic implements Serializable {
    @Id
    @Column(name = "id")
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id ; //编号
    @Column(name = "username")
    private String username ;// 用户名
    @Column(name = "upload_time")
    private Date uploadTime ;// 上传时间
    @Column(name = "upload_location")
    private String uploadLocation ; // 上传图片的客户端位置
    @Column(name = "upload_ip")
    private String uploadIp ; // 上传图片公网ip
    @Column(name = "type")
    private String type ;// 图片类型
    @Column(name = "path")
    private String path ; // 访问路径
    @Column(name = "used")
    private String used ; // 当前是否被使用：0未被使用，1被使用
}

fastDFS配置文件：不是application.yml，application.yml中只需要配置文件大小即可。

微服务common模块获取文件上传地址微服务文件上传下载_微服务common模块获取文件上传地址_05

fdfs_client.conf

#fastDFS的客户端访问配置文件
connect_timeout= 60
network_timeout= 60
charset= UTF-8
#Tracker的Http请求端口
http.tracker_http_port= 8080
tracker_server= 101.200.240.33:22122
# org.csource.common.MyException: getStoreStorage fail, errno code: 2 :
#原因：就是storage容器中ip地址错了
# vim会保存一个 swap,上次修改未保存的就出出现提示，删除swap文件就正常了

注意：项目上传文件连接超时很可能是因为没开放端口：22122和23000. 我的是阿里云服务器，特别还需要手动去阿里控制台去开放这两个端口才能生效。

本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。