Skip to content

Conversation

@ayush00git
Copy link

Why?

As requested in Issue #3130, JavaScript implementation was missing support for unsigned integer types that are already available in other Fory language implementations (Rust, Go, C++, Python). This PR adds complete unsigned integer support for JavaScript.

What does this PR do?

This PR implements schema-based support for unsigned integer types in the Fory JavaScript/TypeScript library:

Added Types

  • uint8
  • uint16
  • uint32
  • uint64
  • var_uint32
  • var_uint64
  • tagged_uint64

Changes Made

1. lib/type.ts

Added 7 unsigned integer types to the InternalSerializerType enum:

export enum InternalSerializerType {
  UINT8,
  UINT16,
  UINT32,
  VAR_UINT32,
  UINT64,
  VAR_UINT64,
  TAGGED_UINT64,
  // ...
}

2. lib/typeInfo.ts

Added Type helper functions for user-facing API:

export const Type = {
  uint8() {
    return TypeInfo.fromNonParam(InternalSerializerType.UINT8 as const, TypeId.UINT8);
  },
  uint16() {
    return TypeInfo.fromNonParam(InternalSerializerType.UINT16 as const, TypeId.UINT16);
  },
  // ...
}

Updated TypeScript type hints for proper type inference:

  • Added unsigned types to HintInput number type union (returns number for 8/16/32-bit, bigint for 64-bit)
  • Added unsigned types to HintResult type union for proper deserialization type inference

3. lib/gen/number.ts

Registered serializers for all 7 unsigned integer types using the existing buildNumberSerializer pattern:

CodegenRegistry.register(InternalSerializerType.UINT8,
  buildNumberSerializer(
    (builder, accessor) => builder.writer.uint8(accessor),
    builder => builder.reader.uint8()
  )
);

CodegenRegistry.register(InternalSerializerType.UINT16,
  buildNumberSerializer(
    (builder, accessor) => builder.writer.uint16(accessor),
    builder => builder.reader.uint16()
  )
);
//....

4. test/number.test.ts

Added comprehensive tests for all unsigned types:

test('should uint8 work', () => {
  const fory = new Fory({ refTracking: true });
  const serializer = fory.registerSerializer(Type.struct({
    typeName: "example.foo"
  }, {
    a: Type.uint8()
  })).serializer;
  const input = fory.serialize({ a: 255 }, serializer);
  const result = fory.deserialize(input);
  expect(result).toEqual({ a: 255 });
});

test('should uint16 work', () => {
  const fory = new Fory({ refTracking: true });
  const serializer = fory.registerSerializer(Type.struct({
    typeName: "example.foo"
  }, {
    a: Type.uint16()
  })).serializer;
  const input = fory.serialize({ a: 65535 }, serializer);
  const result = fory.deserialize(input);
  expect(result).toEqual({ a: 65535 });
});

\\...

Usage Example

import { Fory, Type } from '@apache-fory/fory';

const fory = new Fory();
const serializer = fory.registerSerializer(Type.struct({
  typeName: 'Foo'
}, {
  f1: Type.uint32(),
  f2: Type.varUInt32(),
  f3: Type.uint64()
})).serializer;

const data = { f1: 4294967295, f2: 1000, f3: 18446744073709551615n };
const bytes = fory.serialize(data, serializer);
const result = fory.deserialize(bytes);

Related issues

Does this PR introduce any user-facing change?

  • Does this PR introduce any public API change?

    • Yes: Adds 7 new Type helper functions (Type.uint8(), Type.uint16(), Type.uint32(), Type.varUInt32(), Type.uint64(), Type.varUInt64(), Type.taggedUInt64())
    • These are additive changes only - no breaking changes to existing APIs
  • Does this PR introduce any binary protocol compatibility change?

    • Yes: Adds support for 7 new TypeId values (9-15) for unsigned integers
    • Cross-language compatible: Uses the same TypeId values as other Fory implementations
    • Backward compatible: Existing serialized data remains compatible

Benchmark

N/A - this PR adds new functionality without modifying existing serialization paths. The underlying writer/reader methods uint8(), uint16() were already implemented and optimized; this PR only exposes them through the type system.


@ayush00git
Copy link
Author

Hey @chaokunyang
Please have a look at the PR, and I additionally made some indentation fixes, because i needed some readability at the moment. If you want I can revert them, please do let me know.
Thanks

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for unsigned integer types to the Fory JavaScript/TypeScript library, bringing it into parity with other language implementations (Rust, Go, C++, Python). The implementation adds 7 new unsigned integer types with corresponding serializers, type helpers, and comprehensive tests.

Changes:

  • Added 7 unsigned integer types (uint8, uint16, uint32, var_uint32, uint64, var_uint64, tagged_uint64) to the InternalSerializerType enum
  • Implemented Type helper functions and TypeScript type hints for proper type inference
  • Registered serializers for all unsigned integer types using existing reader/writer methods
  • Added comprehensive test coverage for all new types
  • Normalized code formatting and indentation in typeInfo.ts

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
javascript/packages/fory/lib/type.ts Added 7 unsigned integer type enum values to InternalSerializerType
javascript/packages/fory/lib/typeInfo.ts Added Type helper functions (uint8-taggedUInt64), updated TypeScript type hints for number/bigint inference, and normalized code formatting
javascript/packages/fory/lib/gen/number.ts Registered serializers for all 7 unsigned types using buildNumberSerializer pattern with appropriate reader/writer methods
javascript/test/number.test.ts Added comprehensive tests for all 7 unsigned integer types testing serialization/deserialization with appropriate boundary values

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@ayush00git
Copy link
Author

@chaokunyang
The copilot commented to revert the indentation changes i made, should i revert them? Or is it just a minor change which would work ?

@theweipeng
Copy link
Member

@chaokunyang The copilot commented to revert the indentation changes i made, should i revert them? Or is it just a minor change which would work ?

Great work! It's good to improve the readability. May be you can make some change for the check rules to avoid the CI failure

@ayush00git
Copy link
Author

Great work! It's good to improve the readability. May be you can make some change for the check rules to avoid the CI failure

Hey @theweipeng
Thanks a lot for the review.
Actually the CI tests are only passing if I revert the indentation. Could you suggest me the changes I can make to the check rules to avoid these failures?

@chaokunyang
Copy link
Collaborator

Here is our ci configuration:

  javascript:
    name: JavaScript CI
    strategy:
      matrix:
        node-version: [18, 20, 24]
        os: [ubuntu-latest, macos-latest, windows-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v5
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
          cache-dependency-path: javascript/package-lock.json
          cache: "npm"
      - name: Upgrade npm
        run: npm install -g npm@8
      # node-gyp needs to use python and relies on the distutils module.
      # The distutils module has been removed starting from python 3.12
      # (see https://docs.python.org/3.10/library/distutils.html). Some
      # OS (such as macos-latest) uses python3.12 by default, so python 3.8
      # is used here to avoid this problem.
      - name: Set up Python3.8
        uses: actions/setup-python@v5
        with:
          python-version: 3.8
      - name: Run CI with NodeJS
        run: python ./ci/run_ci.py javascript

You can run tests locally to see why it's failed, or you can look into the error message for how to fix it

@theweipeng
Copy link
Member

Great work! It's good to improve the readability. May be you can make some change for the check rules to avoid the CI failure

Hey @theweipeng Thanks a lot for the review. Actually the CI tests are only passing if I revert the indentation. Could you suggest me the changes I can make to the check rules to avoid these failures?

If you feel hard to chage the CI script, you can revert this part of code. I will change it next PR.

@ayush00git
Copy link
Author

If you feel hard to chage the CI script, you can revert this part of code. I will change it next PR.

Actually i want to learn this thing. I'll give it a look for a while, and if i'm unable to get things done, i'll revert the changes.

@ayush00git
Copy link
Author

Here is our ci configuration:

  javascript:
    name: JavaScript CI
    strategy:
      matrix:
        node-version: [18, 20, 24]
        os: [ubuntu-latest, macos-latest, windows-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v5
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
          cache-dependency-path: javascript/package-lock.json
          cache: "npm"
      - name: Upgrade npm
        run: npm install -g npm@8
      # node-gyp needs to use python and relies on the distutils module.
      # The distutils module has been removed starting from python 3.12
      # (see https://docs.python.org/3.10/library/distutils.html). Some
      # OS (such as macos-latest) uses python3.12 by default, so python 3.8
      # is used here to avoid this problem.
      - name: Set up Python3.8
        uses: actions/setup-python@v5
        with:
          python-version: 3.8
      - name: Run CI with NodeJS
        run: python ./ci/run_ci.py javascript

You can run tests locally to see why it's failed, or you can look into the error message for how to fix it

Hey @chaokunyang
Actually this CI test was already successful, even here. The main issue was with the - python ./ci/run_ci.py format which was showing that errors due to indentation. I just put the file typeInfo.ts in the ignorePatterns and now the tests are passing. Please let me know, if this was a good approach ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[JavaScript] support unsigned number for JavaScript

3 participants