Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

中文标题【MessagePack 的 Jackson 数据格式】

Maven Central Javadoc

本页面中的所有示例程序,请到代码库 https://github.com/cwiki-us-demo/serialize-deserialize-demo-java 中下载进行测试。

Jackson 扩展库是一个易读取,同时是使用  MessagePack 创建的组件库,用于从 jackson-databind API 对数据进行编码。

这个组件扩展了标准的 Jackson streaming API (JsonFactory, JsonParser, JsonGenerator), 同时能够更加无缝的集成更加高级别的数据抽象(数据绑定,树模型和插件扩展)。

更多有关 Jackson-annotations 细节,请参考  https://github.com/FasterXML/jackson-annotations

这个库针对默认 POJO的序列化/反序列化(serialization/deserialization)不能兼容  msgpack-java v0.6 或者早期的版本。

请查看本页中 Advanced usage 的内容。

安装

Maven

Code Block
<dependency>
  <groupId>org.msgpack</groupId>
  <artifactId>jackson-dataformat-msgpack</artifactId>
  <version>(version)</version>
</dependency>

Sbt

Code Block
libraryDependencies += "org.msgpack" % "jackson-dataformat-msgpack" % "(version)"

Gradle

Code Block
repositories {
    mavenCentral()
}

dependencies {
    compile 'org.msgpack:jackson-dataformat-msgpack:(version)'
}


基本使用

在本测试代码中,我们定义了一个 POJO 类,名字为 MessageData,你可以访问下面的链接找到有关这个类的定义。

https://github.com/cwiki-us-demo/serialize-deserialize-demo-java/blob/master/src/main/java/com/insight/demo/serialize/model/msgpack/MessageData.java

POJO 的序列化和反序列化

你仅仅需要对 MessagePackFactory 进行实例化,然后传递参数到 com.fasterxml.jackson.databind.ObjectMapper 的构造方法。

然后你就可以用与 jackson-databind 相同的方法使用 MessagePack 格式化数据。

本测试方法,可以在 https://github.com/cwiki-us-demo/serialize-deserialize-demo-java/blob/master/src/test/java/com/insight/demo/serialize/MessagePackSerializer.java 中找到。

Code Block
languagejava
    /**
     * SerializationPOJO
     */
    @Test
    public void testMessagePackSerializationPOJO() {

        byte[] bytes = new byte[0];
        String uuid = UUID.randomUUID().toString();

        // Instantiate ObjectMapper for MessagePack
        ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());


        MessageData pojo = new MessageData();
        pojo.setUuid(uuid);
        pojo.setName("CWIKI.US");


        try {
            // Serialize a Java object to byte array
            bytes = objectMapper.writeValueAsBytes(pojo);
            logger.debug("Length of Bytes: [{}]", bytes.length);

            // Deserialize the byte array to a Java object
            MessageData deserialized = objectMapper.readValue(bytes, MessageData.class);
            logger.debug("Deserialized Name: [{}]", deserialized.name);

            assertEquals("CWIKI.US", deserialized.name);

        } catch (JsonProcessingException ex) {
            logger.error("Serialize Error", ex);
        } catch (IOException e) {
            e.printStackTrace();
        }

    }


列表(List)的序列化和反序列化

本测试方法,可以在 https://github.com/cwiki-us-demo/serialize-deserialize-demo-java/blob/master/src/test/java/com/insight/demo/serialize/MessagePackSerializer.java 中找到。

我们需要先定义一个对象的 List 列表。

Code Block
    /**
     * SerializationList
     */
    @Test
    public void testMessagePackSerializationList() {

        byte[] bytes = new byte[0];
        String uuid = UUID.randomUUID().toString();

        // Instantiate ObjectMapper for MessagePack
        ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());

        List<MessageData> objList = MockDataUtils.getMessageDataList(9);

        try {
            // Serialize a Java object to byte array
            bytes = objectMapper.writeValueAsBytes(objList);
            logger.debug("Length of Bytes: [{}]", bytes.length);

            // Deserialize the byte array to a Java object
            // Deserialize the byte array to a List
            List<MessageData> deserialized = objectMapper.readValue(bytes, new TypeReference<List<MessageData>>() {
            });
            logger.debug("Deserialized List Count: [{}]", deserialized.size());
            logger.debug("List index 0: [{}]", deserialized.get(0).name);

        } catch (JsonProcessingException ex) {
            logger.error("Serialize Error", ex);
        } catch (IOException e) {
            e.printStackTrace();
        }

    }


Map 的序列化和反序列化

本测试方法,可以在 https://github.com/cwiki-us-demo/serialize-deserialize-demo-java/blob/master/src/test/java/com/insight/demo/serialize/MessagePackSerializer.java 中找到。

我们需要定义测试需要的 MAP,定义 Map 的方法你可以在下面的测试程序中找到。

Code Block
    /**
     * SerializationMap
     */
    @Test
    public void testMessagePackSerializationMap() {

        byte[] bytes = new byte[0];
        String uuid_a = UUID.randomUUID().toString();
        String uuid_b = UUID.randomUUID().toString();

        // Instantiate ObjectMapper for MessagePack
        ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());

        Map<String, MessageData> map = new HashMap<>();
        MessageData messageData = new MessageData();

        // Element A in MAP
        messageData.setUuid(UUID.randomUUID().toString());
        messageData.setName("CWIKI.US - A");
        map.put(uuid_a, messageData);

        // Element B in MAP
        messageData = new MessageData();
        messageData.setUuid(UUID.randomUUID().toString());
        messageData.setName("CWIKI.US - B");
        map.put(uuid_b, messageData);


        try {
            // Serialize a Java object to byte array
            bytes = objectMapper.writeValueAsBytes(map);
            logger.debug("Length of Bytes: [{}]", bytes.length);

            // Deserialize the byte array to a MAP
            Map<String, MessageData> deserialized = objectMapper.readValue(bytes, new TypeReference<Map<String, MessageData>>() {
            });
            logger.debug("Deserialized MAP Count: [{}]", deserialized.size());
            logger.debug("MAP index 0: [{}]", deserialized.get(uuid_a).getName());

            assertEquals("CWIKI.US - A", deserialized.get(uuid_a).getName());

        } catch (JsonProcessingException ex) {
            logger.error("Serialize Error", ex);
        } catch (IOException e) {
            e.printStackTrace();
        }


    }


针对多语言的序列化和反序列化示例

Java 语言的序列化

Code Block
languagejava
  // Serialize
  Map<String, Object> obj = new HashMap<String, Object>();
  obj.put("foo", "hello");
  obj.put("bar", "world");
  byte[] bs = objectMapper.writeValueAsBytes(obj);
  // bs => [-126, -93, 102, 111, 111, -91, 104, 101, 108, 108, 111,
  //        -93, 98, 97, 114, -91, 119, 111, 114, 108, 100]


Ruby 语言的序列化

Code Block
  require 'msgpack'

  # Deserialize
  xs = [-126, -93, 102, 111, 111, -91, 104, 101, 108, 108, 111,
        -93, 98, 97, 114, -91, 119, 111, 114, 108, 100]
  MessagePack.unpack(xs.pack("C*"))
  # => {"foo"=>"hello", "bar"=>"world"}

  # Serialize
  ["zero", 1, 2.0, nil].to_msgpack.unpack('C*')
  # => [148, 164, 122, 101, 114, 111, 1, 203, 64, 0, 0, 0, 0, 0, 0, 0, 192]


Java 语言的反序列化

Code Block
  // Deserialize
  bs = new byte[] {(byte) 148, (byte) 164, 122, 101, 114, 111, 1,
                   (byte) 203, 64, 0, 0, 0, 0, 0, 0, 0, (byte) 192};
  TypeReference<List<Object>> typeReference = new TypeReference<List<Object>>(){};
  List<Object> xs = objectMapper.readValue(bs, typeReference);
  // xs => [zero, 1, 2.0, null]


高级使用

下面为 MessagePack 的一些高级使用。

序列化和反序列化 POJO 为 MessagePack 的数组类型用来与 msgpack-java:0.6 保持兼容性

在 msgpack-java 0.6 或者早期的版本中,POJO 在 MessagePack 中被序列化和反序列化为数组变量。

变量的顺序是基于 Java 类中变量的内部顺序了,这种本来是一种原生的序列化方法,但是会导致一些问题。导致这些问题的原因是在 Java 对象的内部变量的顺序与 Java 实现的顺序不能保证完全的一致。

从另外一个角度来看,使用 jackson-databind 进行的序列化和反序列化方式是基于 POJO 的 Key-Value 对的。因此在 jackson-dataformat-msgpack 与 POJO 处理的方式是相同的。因此,这就导致了与 msgpack-java:0.6 或者早期的版本在对 POJO 进行序列化和反序列化的时候不兼容。

如果你希望在新的版本中也使用与 msgpack-java:0.6 或者早期版本相同的处理 POJO 的方法,你可以使用 JsonArrayFormat。具体使用方法如下所示:

Code Block
  ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());
  objectMapper.setAnnotationIntrospector(new JsonArrayFormat());



在不关闭输出流(output stream)的情况下序列化多变量

com.fasterxml.jackson.databind.ObjectMapper 在默认的情况下在写出输入后将会关闭输出流(output stream)。

如果你希望序列化多值变量在同一个输出流的情况下,你不希望在输出完一个就关闭输出流,你可以设置  JsonGenerator.Feature.AUTO_CLOSE_TARGET 参数为 False

本测试方法,可以在 https://github.com/cwiki-us-demo/serialize-deserialize-demo-java/blob/master/src/test/java/com/insight/demo/serialize/MessagePackSerializer.java 中找到。

Code Block
    /**
     * Serialization Not Close output stream
     */
    @Test
    public void testMessagePackSerializationNotCloseOutputStream() {
        logger.debug("testMessagePackSerializationNotCloseOutputStream");

        try {
            File tempFile = File.createTempFile("messagepack-", "-cwiki.us");

            OutputStream out = new FileOutputStream(tempFile);
            ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());
            objectMapper.configure(JsonGenerator.Feature.AUTO_CLOSE_TARGET, false);

            objectMapper.writeValue(out, 1);
            objectMapper.writeValue(out, "two");
            objectMapper.writeValue(out, 3.14);
            out.close();

            MessageUnpacker unpacker = MessagePack.newDefaultUnpacker(new FileInputStream(tempFile));
            System.out.println(unpacker.unpackInt());      // => 1
            System.out.println(unpacker.unpackString());   // => two
            System.out.println(unpacker.unpackFloat());    // => 3.14

            tempFile.deleteOnExit();
        } catch (IOException ex) {
            logger.error("Serialize Error", ex);
        }
    }


在不关闭输入流(input stream)的情况下反序列化多变量

com.fasterxml.jackson.databind.ObjectMapper 在读取输入流变量的时候默认的将会关闭输入流。

如果你不希望关闭输入流,你可以设置 JsonParser.Feature.AUTO_CLOSE_SOURCE 参数为 false。

本测试方法,可以在 https://github.com/cwiki-us-demo/serialize-deserialize-demo-java/blob/master/src/test/java/com/insight/demo/serialize/MessagePackSerializer.java 中找到。


Code Block
    /**
     * Serialization Not Close input stream
     */
    @Test
    public void testMessagePackSerializationNotCloseInputStream() {
        logger.debug("testMessagePackSerializationNotCloseInputStream");

        try {
            File tempFile = File.createTempFile("messagepack-", "-cwiki.us");

            MessagePacker packer = MessagePack.newDefaultPacker(new FileOutputStream(tempFile));
            packer.packInt(42);
            packer.packString("Hello");
            packer.close();

            FileInputStream in = new FileInputStream(tempFile);
            ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());
            objectMapper.configure(JsonParser.Feature.AUTO_CLOSE_SOURCE, false);
            System.out.println(objectMapper.readValue(in, Integer.class));
            System.out.println(objectMapper.readValue(in, String.class));
            in.close();

            tempFile.deleteOnExit();
        } catch (IOException ex) {
            logger.error("Serialize Error", ex);
        }
    }


不使用 str8 数据类型的序列化

老的 msgpack-java(例如 0.6.7)并不支持 MessagePack str8 数据类型。

当你的希望的你的应用程序需要支持老的版本的话,你需要禁用这个数据类型,例如使用下面的语句:

Code Block
  MessagePack.PackerConfig config = new MessagePack.PackerConfig().withStr8FormatSupport(false);
  ObjectMapper mapperWithConfig = new ObjectMapper(new MessagePackFactory(config));
  // This string is serialized as bin8 type
  byte[] resultWithoutStr8Format = mapperWithConfig.writeValueAsBytes(str8LengthString);


在 Map 中不使用 String 为 Key 的序列化

当你希望在 Map 中不使用 String 为 Key,那么你需要使用 MessagePackKeySerializer 来为 key 进行序列化。

本测试方法,可以在 https://github.com/cwiki-us-demo/serialize-deserialize-demo-java/blob/master/src/test/java/com/insight/demo/serialize/MessagePackSerializer.java 中找到。


Code Block
    /**
     * testMessagePackSerializationMapKey
     */
    @Test
    @JsonSerialize(keyUsing = MessagePackKeySerializer.class)
    public void testMessagePackSerializationMapKey() {
        logger.debug("testMessagePackSerializationNotCloseInputStream");

        byte[] bytes = new byte[0];
        Integer uuid_a = 101;
        Integer uuid_b = 102;

        // Instantiate ObjectMapper for MessagePack
        ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());

        Map<Integer, MessageData> map = new HashMap<>();
        MessageData messageData = new MessageData();

        // Element A in MAP
        messageData.setUuid(UUID.randomUUID().toString());
        messageData.setName("CWIKI.US - A");
        map.put(uuid_a, messageData);

        // Element B in MAP
        messageData = new MessageData();
        messageData.setUuid(UUID.randomUUID().toString());
        messageData.setName("CWIKI.US - B");
        map.put(uuid_b, messageData);


        try {
            // Serialize a Java object to byte array
            bytes = objectMapper.writeValueAsBytes(map);
            logger.debug("Length of Bytes: [{}]", bytes.length);

            // Deserialize the byte array to a MAP
            Map<String, MessageData> deserialized = objectMapper.readValue(bytes, new TypeReference<Map<Integer, MessageData>>() {
            });
            logger.debug("Deserialized MAP Count: [{}]", deserialized.size());
            logger.debug("MAP index 0: [{}]", deserialized.get(uuid_a).getName());

            assertEquals("CWIKI.US - A", deserialized.get(uuid_a).getName());

        } catch (JsonProcessingException ex) {
            logger.error("Serialize Error", ex);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

Deserialize extension types with


使用  ExtensionTypeCustomDeserializers来反序列化扩展类型

ExtensionTypeCustomDeserializers helps you to deserialize extension types easily.

With target Java class

 能够帮助你更加容易的反序列化扩展类型。


使用 target Java 类


Code Block
NestedListComplexPojo parent = new NestedListComplexPojo();

  parent.children = Arrays.asList(new TinyPojo("Foo"), new TinyPojo("Bar"));

  // In this application, extension type 17 is used for NestedListComplexPojo
  byte[] bytes;
  {
      // This ObjectMapper is just for temporary serialization
      ObjectMapper tempObjectMapper = new ObjectMapper(new MessagePackFactory());
      ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
      MessagePacker packer = MessagePack.newDefaultPacker(outputStream);

      byte[] extBytes = tempObjectMapper.writeValueAsBytes(parent);
      packer.packExtensionTypeHeader((byte) 17, extBytes.length);
      packer.addPayload(extBytes);
      packer.close();

      bytes = outputStream.toByteArray();
  }

  // Register the type and the class to ExtensionTypeCustomDeserializers
  ExtensionTypeCustomDeserializers extTypeCustomDesers = new ExtensionTypeCustomDeserializers();
  extTypeCustomDesers.addTargetClass((byte) 17, NestedListComplexPojo.class);
  ObjectMapper objectMapper = new ObjectMapper(
          new MessagePackFactory().setExtTypeCustomDesers(extTypeCustomDesers));

  System.out.println(objectMapper.readValue(bytes, Object.class));
    // => NestedListComplexPojo{children=[TinyPojo{name='Foo'}, TinyPojo{name='Bar'}]}
With type reference


使用类型参考


Code Block
  
Map<String
Map<String, 
Integer>
Integer> map = new HashMap<>();
  map.put("one", 1);
  map.put("two", 2);

  // In this application, extension type 31 is used for Map<String, Integer>
  byte[] bytes;
  {
      // Same as above
        :
      packer.packExtensionTypeHeader((byte) 31, extBytes.length);
        :
  }

  // Register the type and the type reference to ExtensionTypeCustomDeserializers
  ExtensionTypeCustomDeserializers extTypeCustomDesers = new ExtensionTypeCustomDeserializers();
  extTypeCustomDesers.addTargetTypeReference((byte) 31,
  	      new 
TypeReference<Map<String
TypeReference<Map<String, 
Integer>>
Integer>>() {});
  ObjectMapper objectMapper = new ObjectMapper(
          new MessagePackFactory().setExtTypeCustomDesers(extTypeCustomDesers));

  System.out.println(objectMapper.readValue(bytes, Object.class));
    // => {one=1, two=2}


自定义反序列化

With custom deserializer


Code Block
  // In this application, extension type 59 is used for byte[]
  byte[] bytes;
  {
      // This ObjectMapper is just for temporary serialization
      ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
      MessagePacker packer = MessagePack.newDefaultPacker(outputStream);

      packer.packExtensionTypeHeader((byte) 59, hexspeak.length);
      packer.addPayload(hexspeak);
      packer.close();

      bytes = outputStream.toByteArray();
  }

  // Register the type and a deserializer to ExtensionTypeCustomDeserializers
  ExtensionTypeCustomDeserializers extTypeCustomDesers = new ExtensionTypeCustomDeserializers();
  extTypeCustomDesers.addCustomDeser((byte) 59, data -> {
      if (Arrays.equals(data,
                new byte[] {(byte) 0xCA, (byte) 0xFE, (byte) 0xBA, (byte) 0xBE})) {
          return "Java";
      }
      return "Not Java";
  }
  );
  ObjectMapper objectMapper = new ObjectMapper(
          new MessagePackFactory().setExtTypeCustomDesers(extTypeCustomDesers));

  System.out.println(objectMapper.readValue(bytes, Object.class));
    // => Java