Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Final updates to runtime and codegen fixes #352

Merged
merged 13 commits into from
Feb 27, 2025
26 changes: 21 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Supports only version 3 of the Protocol Buffers language.

### Features
- Idiomatic and concise [Kotlin builder DSL](#generated-code)
- Protokt-specific options: [non-null types (dangerous)](#non-null-fields),
- Protokt-specific options: [non-null accessors](#non-null-accessors),
[wrapper types](#wrapper-types),
[interface implementation](#interface-implementation),
and more
Expand Down Expand Up @@ -636,8 +636,9 @@ dependencies {
```

Wrapper types that wrap protobuf messages are nullable. For example,
`java.time.Instant` wraps the well-known type `google.protobuf.Timestamp`. They
can be made non-nullable by using the non-null option described below.
`java.time.Instant` wraps the well-known type `google.protobuf.Timestamp`. You
can generate non-null accessors with the `generate_non_null_accessor` option
described below.

Wrapper types that wrap protobuf primitives, for example `java.util.UUID`
which wraps `bytes`, are nullable when they cannot wrap their wrapped type's
Expand All @@ -659,7 +660,8 @@ google.protobuf.BytesValue nullable_uuid = 3 [
];
```
This behavior can be overridden with the [`non_null` option](#non-null-fields).
As for message types, you can generate non-null accessors with the
`generate_non_null_accessor` option.
Wrapper types can be repeated:
Expand Down Expand Up @@ -687,7 +689,7 @@ _N.b. Well-known type nullability is implemented with
for each message defined in
[wrappers.proto](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/wrappers.proto)._
### Non-null fields
### Non-null accessors
If a message has no meaning whatsoever when a particular non-scalar field is
missing, you can emulate proto2's `required` key word by using the
`(protokt.v1.property).generate_non_null_accessor` option:
Expand All @@ -705,6 +707,20 @@ message NonNullSampleMessage {
Generated code will include a non-null accessor prefixed with `require`, so the field can be referenced
without using Kotlin's `!!`.
This option also works on oneof fields:
```protobuf
message Sample {}
message NonNullSampleMessage {
oneof non_null_sample {
option (protokt.v1.oneof).generate_non_null_accessor = true;
Sample sample = 1;
}
}
```
### Interface implementation
#### Messages
Expand Down
2 changes: 1 addition & 1 deletion buildSrc/src/main/kotlin/RemotePublishing.kt
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ fun Project.enablePublishing(defaultJars: Boolean = true) {
configure(KotlinMultiplatform(JavadocJar.Empty()))
}
pluginManager.withPlugin(KotlinPlugins.JVM) {
configure(KotlinJvm(JavadocJar.Empty(), true))
configure(KotlinJvm(JavadocJar.Empty()))
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the default value.

}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ allprojects {
targetExclude("buildSrc/build/**")
licenseHeaderFile(
rootProject.file("gradle/license-header-c-style"),
"(package |@file|import |fun )|buildscript |plugins |subprojects |spotless "
"(package |@file|import |fun )|buildscript |plugins |subprojects |spotless |group ="
)
}

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (c) 2020 Toast, Inc.
* Copyright (c) 2022 Toast, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand All @@ -13,6 +13,4 @@
* limitations under the License.
*/

subprojects {
group = "${rootProject.group}.thirdparty"
}
group = "com.toasttab.protokt.thirdparty"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was getting clobbered by the common-conventions plugin - we've been publishing to the wrong group since that change went in.

Original file line number Diff line number Diff line change
Expand Up @@ -593,11 +593,12 @@ public final class protokt/v1/MethodOptions$Deserializer : protokt/v1/AbstractDe

public final class protokt/v1/OneofOptions : protokt/v1/AbstractMessage {
public static final field Deserializer Lprotokt/v1/OneofOptions$Deserializer;
public synthetic fun <init> (Ljava/lang/String;Ljava/lang/String;Lprotokt/v1/UnknownFieldSet;Lkotlin/jvm/internal/DefaultConstructorMarker;)V
public synthetic fun <init> (ZLjava/lang/String;Ljava/lang/String;Lprotokt/v1/UnknownFieldSet;Lkotlin/jvm/internal/DefaultConstructorMarker;)V
public final fun copy (Lkotlin/jvm/functions/Function1;)Lprotokt/v1/OneofOptions;
public static fun deserialize (Lprotokt/v1/Reader;)Lprotokt/v1/OneofOptions;
public fun equals (Ljava/lang/Object;)Z
public final fun getDeprecationMessage ()Ljava/lang/String;
public final fun getGenerateNonNullAccessor ()Z
public final fun getImplements ()Ljava/lang/String;
public final fun getUnknownFields ()Lprotokt/v1/UnknownFieldSet;
public fun hashCode ()I
Expand All @@ -611,9 +612,11 @@ public final class protokt/v1/OneofOptions$Builder {
public fun <init> ()V
public final fun build ()Lprotokt/v1/OneofOptions;
public final fun getDeprecationMessage ()Ljava/lang/String;
public final fun getGenerateNonNullAccessor ()Z
public final fun getImplements ()Ljava/lang/String;
public final fun getUnknownFields ()Lprotokt/v1/UnknownFieldSet;
public final fun setDeprecationMessage (Ljava/lang/String;)V
public final fun setGenerateNonNullAccessor (Z)V
public final fun setImplements (Ljava/lang/String;)V
public final fun setUnknownFields (Lprotokt/v1/UnknownFieldSet;)V
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,19 @@ extend google.protobuf.FieldOptions {
}

message OneofOptions {
reserved 1;
// Generates a non-nullable accessor for a oneof field.
//
// For example:
//
// message Message {
// oneof some_field_name {
// option (protokt.v1.oneof).generate_non_null_accessor = true;
//
// string id = 1;
// }
// }
//
bool generate_non_null_accessor = 1;

// Make the sealed class implement an interface, enforcing the presence of a
// property in each possible variant. Scoping rules are the same as those
Expand Down
10 changes: 5 additions & 5 deletions extensions/protokt-extensions/api/protokt-extensions.api
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,6 @@ public final class protokt/v1/Inet_socket_address_descriptorsKt {
public static final fun getDescriptor (Lprotokt/v1/InetSocketAddress$Deserializer;)Lprotokt/v1/google/protobuf/Descriptor;
}

public final class protokt/v1/ProtoktProtos {
public static final field INSTANCE Lprotokt/v1/ProtoktProtos;
public final fun getDescriptor ()Lprotokt/v1/google/protobuf/FileDescriptor;
}

public final class protokt/v1/Protokt_descriptorsKt {
public static final fun getDescriptor (Lprotokt/v1/EnumOptions$Deserializer;)Lprotokt/v1/google/protobuf/Descriptor;
public static final fun getDescriptor (Lprotokt/v1/EnumValueOptions$Deserializer;)Lprotokt/v1/google/protobuf/Descriptor;
Expand All @@ -48,3 +43,8 @@ public final class protokt/v1/Protokt_descriptorsKt {
public static final fun getDescriptor (Lprotokt/v1/ServiceOptions$Deserializer;)Lprotokt/v1/google/protobuf/Descriptor;
}

public final class protokt/v1/protokt_file_descriptor {
public static final field INSTANCE Lprotokt/v1/protokt_file_descriptor;
public final fun getDescriptor ()Lprotokt/v1/google/protobuf/FileDescriptor;
}

Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ internal object Implements {
}

private fun TypeSpec.Builder.delegateProperties(msg: Message, ctx: Context, canonicalName: String, fieldName: String) {
val fieldsByName = msg.fields.filterIsInstance<StandardField>().associateBy { it.fieldName }
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't exclude overrides that are oneofs.

val fieldsByName = msg.fields.associateBy { it.fieldName }

val interfaceFields =
ctx.info.context.classLookup
Expand All @@ -77,7 +77,7 @@ internal object Implements {
field.name,
(field.returnType.classifier as KClass<*>).asTypeName()
.let {
if (assumeNotNull) {
if (assumeNotNull && !field.returnType.isMarkedNullable) {
Copy link
Collaborator Author

@andrewparmet andrewparmet Feb 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't delegate a field and mark it as non-nullable if the delegated field is nullable in the interface.

it
} else {
it.copy(nullable = true)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -159,24 +159,24 @@ private class MessageGenerator(
.build()
)
}
if (property.overrides) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the name of a field is the same as the field on the interface and we generate a non-null accessor for it, the override needs to be on the generated delegating property, not the non-null accessor.

addModifiers(KModifier.OVERRIDE)
}
initializer(property.name)
}.build(),
PropertySpec.builder(nonNullPropName(property.name), property.propertyType.copy(nullable = false)).apply {
getter(
FunSpec.getterBuilder()
.addCode("return ${dereferenceNullableBackingProperty(property.name)}")
.addCode("return ${dereferenceNullableBackingProperty(property.name, property.oneof)}")
.build()
)
if (property.overrides) {
addModifiers(KModifier.OVERRIDE)
}
property.documentation?.let { addKdoc(formatDoc(it)) }
handleDeprecation(property.deprecation)
}.build()
)

private fun dereferenceNullableBackingProperty(propName: String) =
"requireNotNull($propName) { \"$propName is assumed non-null with (protokt.property).generate_non_null_accessor but was null\" }".bindSpaces()
private fun dereferenceNullableBackingProperty(propName: String, oneof: Boolean) =
"requireNotNull($propName) { \"$propName is assumed non-null with (protokt.v1.${if (oneof) "oneof" else "property"}).generate_non_null_accessor but was null\" }".bindSpaces()

private fun TypeSpec.Builder.handleMessageSize(propertySpecs: List<PropertySpec>) {
addProperty(generateMessageSize(msg, propertySpecs, ctx))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,9 @@ import protokt.v1.reflect.FieldType

internal object Nullability {
val Field.generateNonNullAccessor
get() = this is StandardField && options.protokt.generateNonNullAccessor
get() =
(this is StandardField && options.protokt.generateNonNullAccessor) ||
(this is Oneof && options.protokt.generateNonNullAccessor)

val Field.nullable
get() = isKotlinRepresentationNullable
Expand All @@ -43,11 +45,7 @@ internal object Nullability {
type !in setOf(FieldType.Message, FieldType.Enum)

fun propertyType(o: Oneof) =
if (o.generateNonNullAccessor) {
Copy link
Collaborator Author

@andrewparmet andrewparmet Feb 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was previously non-functional but now that it can sometimes be true, it was causing compilation failure.

o.className
} else {
o.className.copy(nullable = true)
}
o.className.copy(nullable = true)

fun propertyType(f: StandardField, type: TypeName, wrapperRequiresNullability: Boolean) =
if (f.nullable || wrapperRequiresNullability) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ private class PropertyAnnotator(
PropertyInfo(
name = field.fieldName,
propertyType = propertyType(field),
generateNullableBackingProperty = field.generateNonNullAccessor,
deserializeType = field.className.copy(nullable = true),
builderPropertyType = field.className.copy(nullable = true),
defaultValue = field.defaultValue(ctx, false),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,6 @@ private fun generateFdpObjectNames(files: List<FileDescriptorProto>): Map<String
Pair(
fdp.name,
fdp.fileOptions.protokt.fileDescriptorObjectName.takeIf { it.isNotEmpty() }
?: fdp.fileOptions.default.javaOuterClassname.takeIf { it.isNotEmpty() }
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore all Java options.

?: (fdp.name.substringBefore(".proto").substringAfterLast('/') + "_file_descriptor")
)
}
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ import kotlin.Unit
* { "@type": "type.googleapis.com/google.protobuf.Duration", "value": "1.212s"
* }
*/
@Deprecated("for backwards compatibility only")
@KtGeneratedMessage("google.protobuf.Any")
class Any private constructor(
/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ import kotlin.collections.MutableList
* They are also sometimes simply referred to as "APIs" in other contexts, such as the name of this
* message itself. See https://cloud.google.com/apis/design/glossary for detailed terminology.
*/
@Deprecated("for backwards compatibility only")
@KtGeneratedMessage("google.protobuf.Api")
class Api private constructor(
/**
Expand Down Expand Up @@ -281,6 +282,7 @@ class Api private constructor(
/**
* Method represents a method of an API interface.
*/
@Deprecated("for backwards compatibility only")
@KtGeneratedMessage("google.protobuf.Method")
class Method private constructor(
/**
Expand Down Expand Up @@ -538,6 +540,7 @@ class Method private constructor(
* returns (Acl) { option (google.api.http).get = "/v2/acls/{resource=**}:getAcl"; }
* ... }
*/
@Deprecated("for backwards compatibility only")
@KtGeneratedMessage("google.protobuf.Mixin")
class Mixin private constructor(
/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ import kotlin.collections.MutableList
/**
* The version number of protocol compiler.
*/
@Deprecated("for backwards compatibility only")
@KtGeneratedMessage("google.protobuf.compiler.Version")
class Version private constructor(
val major: Int?,
Expand Down Expand Up @@ -345,6 +346,7 @@ class CodeGeneratorRequest private constructor(
/**
* The plugin writes an encoded CodeGeneratorResponse to stdout.
*/
@Deprecated("for backwards compatibility only")
@KtGeneratedMessage("google.protobuf.compiler.CodeGeneratorResponse")
class CodeGeneratorResponse private constructor(
/**
Expand Down
Loading