-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix ConstructorConstructor creating mismatching Collection and Map instances #2068
base: main
Are you sure you want to change the base?
Changes from all commits
5b83a03
d5d7cdd
c6aa3b1
2b6fa06
6a63611
9952bda
1f496d7
f8ca948
e70a754
86b13f2
ae94ae8
0bb06e1
8c38d6e
86dcb42
c13f459
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -36,15 +36,9 @@ | |
import java.util.LinkedHashSet; | ||
import java.util.List; | ||
import java.util.Map; | ||
import java.util.Queue; | ||
import java.util.Set; | ||
import java.util.SortedMap; | ||
import java.util.SortedSet; | ||
import java.util.TreeMap; | ||
import java.util.TreeSet; | ||
import java.util.concurrent.ConcurrentHashMap; | ||
import java.util.concurrent.ConcurrentMap; | ||
import java.util.concurrent.ConcurrentNavigableMap; | ||
import java.util.concurrent.ConcurrentSkipListMap; | ||
|
||
/** Returns a function that can construct an instance of a requested type. */ | ||
|
@@ -94,7 +88,19 @@ static String checkInstantiable(Class<?> c) { | |
return null; | ||
} | ||
|
||
/** Calls {@link #get(TypeToken, boolean)}, and allows usage of JDK Unsafe. */ | ||
public <T> ObjectConstructor<T> get(TypeToken<T> typeToken) { | ||
return get(typeToken, true); | ||
} | ||
|
||
/** | ||
* Retrieves an object constructor for the given type. | ||
* | ||
* @param typeToken type for which a constructor should be retrieved | ||
* @param allowUnsafe whether to allow usage of JDK Unsafe; has no effect if {@link #useJdkUnsafe} | ||
* is false | ||
*/ | ||
public <T> ObjectConstructor<T> get(TypeToken<T> typeToken, boolean allowUnsafe) { | ||
Type type = typeToken.getType(); | ||
Class<? super T> rawType = typeToken.getRawType(); | ||
|
||
|
@@ -142,12 +148,19 @@ public <T> ObjectConstructor<T> get(TypeToken<T> typeToken) { | |
}; | ||
} | ||
|
||
if (!allowUnsafe) { | ||
String message = | ||
"Unable to create instance of " | ||
+ rawType | ||
+ "; Register an InstanceCreator or a TypeAdapter for this type."; | ||
return () -> { | ||
throw new JsonIOException(message); | ||
}; | ||
} | ||
|
||
// Consider usage of Unsafe as reflection, so don't use if BLOCK_ALL | ||
// Additionally, since it is not calling any constructor at all, don't use if BLOCK_INACCESSIBLE | ||
if (filterResult == FilterResult.ALLOW) { | ||
// finally try unsafe | ||
return newUnsafeAllocator(rawType); | ||
} else { | ||
if (filterResult != FilterResult.ALLOW) { | ||
String message = | ||
"Unable to create instance of " | ||
+ rawType | ||
|
@@ -158,6 +171,9 @@ public <T> ObjectConstructor<T> get(TypeToken<T> typeToken) { | |
throw new JsonIOException(message); | ||
}; | ||
} | ||
|
||
// finally try unsafe | ||
return newUnsafeAllocator(rawType); | ||
} | ||
|
||
/** | ||
|
@@ -290,7 +306,6 @@ private static <T> ObjectConstructor<T> newDefaultConstructor( | |
} | ||
|
||
/** Constructors for common interface types like Map and List and their subtypes. */ | ||
@SuppressWarnings("unchecked") // use runtime checks to guarantee that 'T' is what it is | ||
private static <T> ObjectConstructor<T> newDefaultImplementationConstructor( | ||
Type type, Class<? super T> rawType) { | ||
|
||
|
@@ -303,33 +318,84 @@ private static <T> ObjectConstructor<T> newDefaultImplementationConstructor( | |
*/ | ||
|
||
if (Collection.class.isAssignableFrom(rawType)) { | ||
if (SortedSet.class.isAssignableFrom(rawType)) { | ||
return () -> (T) new TreeSet<>(); | ||
} else if (Set.class.isAssignableFrom(rawType)) { | ||
return () -> (T) new LinkedHashSet<>(); | ||
} else if (Queue.class.isAssignableFrom(rawType)) { | ||
return () -> (T) new ArrayDeque<>(); | ||
} else { | ||
return () -> (T) new ArrayList<>(); | ||
} | ||
@SuppressWarnings("unchecked") | ||
ObjectConstructor<T> constructor = (ObjectConstructor<T>) newCollectionConstructor(rawType); | ||
return constructor; | ||
} | ||
|
||
if (Map.class.isAssignableFrom(rawType)) { | ||
if (ConcurrentNavigableMap.class.isAssignableFrom(rawType)) { | ||
return () -> (T) new ConcurrentSkipListMap<>(); | ||
} else if (ConcurrentMap.class.isAssignableFrom(rawType)) { | ||
return () -> (T) new ConcurrentHashMap<>(); | ||
} else if (SortedMap.class.isAssignableFrom(rawType)) { | ||
return () -> (T) new TreeMap<>(); | ||
} else if (type instanceof ParameterizedType | ||
&& !String.class.isAssignableFrom( | ||
TypeToken.get(((ParameterizedType) type).getActualTypeArguments()[0]).getRawType())) { | ||
return () -> (T) new LinkedHashMap<>(); | ||
} else { | ||
return () -> (T) new LinkedTreeMap<>(); | ||
} | ||
@SuppressWarnings("unchecked") | ||
ObjectConstructor<T> constructor = (ObjectConstructor<T>) newMapConstructor(type, rawType); | ||
return constructor; | ||
} | ||
|
||
// Unsupported type; try other means of creating constructor | ||
return null; | ||
Comment on lines
+332
to
+333
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In #419 (comment) the concern was mentioned that the fallback will be Not sure if we should throw an exception then in case of Collection or Map? Or we could maybe add an There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @eamonnmcmanus, do you think this is something to consider? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure I understand the concern here. If we reach this point then There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Seems like there is a bug with GitHub and it is not showing the comment I was referring to above; here is an archived version:
When users define custom I have pushed a commit now which prevents Unsafe usage in this case; I hope that is fine. If you don't think it should be included please let me know and I revert it again. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK, I think I understand the concern now. It is not about this statement particularly, but the fact that the earlier We don't see this issue with Guava's |
||
} | ||
|
||
private static ObjectConstructor<? extends Collection<? extends Object>> newCollectionConstructor( | ||
Class<?> rawType) { | ||
|
||
// First try List implementation | ||
if (rawType.isAssignableFrom(ArrayList.class)) { | ||
return () -> new ArrayList<>(); | ||
} | ||
// Then try Set implementation | ||
else if (rawType.isAssignableFrom(LinkedHashSet.class)) { | ||
return () -> new LinkedHashSet<>(); | ||
} | ||
// Then try SortedSet / NavigableSet implementation | ||
else if (rawType.isAssignableFrom(TreeSet.class)) { | ||
return () -> new TreeSet<>(); | ||
} | ||
// Then try Queue implementation | ||
else if (rawType.isAssignableFrom(ArrayDeque.class)) { | ||
return () -> new ArrayDeque<>(); | ||
} | ||
|
||
// Was unable to create matching Collection constructor | ||
return null; | ||
} | ||
|
||
private static boolean hasStringKeyType(Type mapType) { | ||
// If mapType is not parameterized, assume it might have String as key type | ||
if (!(mapType instanceof ParameterizedType)) { | ||
return true; | ||
} | ||
|
||
Type[] typeArguments = ((ParameterizedType) mapType).getActualTypeArguments(); | ||
if (typeArguments.length == 0) { | ||
return false; | ||
} | ||
return $Gson$Types.getRawType(typeArguments[0]) == String.class; | ||
} | ||
|
||
private static ObjectConstructor<? extends Map<? extends Object, Object>> newMapConstructor( | ||
Type type, Class<?> rawType) { | ||
// First try Map implementation | ||
/* | ||
* Legacy special casing for Map<String, ...> to avoid DoS from colliding String hashCode | ||
* values for older JDKs; use own LinkedTreeMap<String, Object> instead | ||
*/ | ||
if (rawType.isAssignableFrom(LinkedTreeMap.class) && hasStringKeyType(type)) { | ||
return () -> new LinkedTreeMap<>(); | ||
} else if (rawType.isAssignableFrom(LinkedHashMap.class)) { | ||
return () -> new LinkedHashMap<>(); | ||
} | ||
// Then try SortedMap / NavigableMap implementation | ||
else if (rawType.isAssignableFrom(TreeMap.class)) { | ||
return () -> new TreeMap<>(); | ||
} | ||
// Then try ConcurrentMap implementation | ||
else if (rawType.isAssignableFrom(ConcurrentHashMap.class)) { | ||
return () -> new ConcurrentHashMap<>(); | ||
} | ||
// Then try ConcurrentNavigableMap implementation | ||
else if (rawType.isAssignableFrom(ConcurrentSkipListMap.class)) { | ||
return () -> new ConcurrentSkipListMap<>(); | ||
} | ||
|
||
// Was unable to create matching Map constructor | ||
return null; | ||
} | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems all other
fromJson
methods delegate to this one, so I have added the type check (which was previouslyPrimitives.wrap(classOfT).cast(object)
in the otherfromJson
methods) here instead.However, this method did not actually have this check before.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I ran this change against all of Google's internal tests and found a lot of failures with this exception, or with an exception about not being able to instantiate the abstract class
com.google.common.collect.ImmutableList
. In both cases I think the code in question should be fixed, because it is relying on accidental behaviour. A pattern I saw quite often was this:This is using auto-value-gson to serialize and deserialize AutoValue classes. It works even if no
TypeAdapterFactory
has been registered forImmutableList
, because Gson deserializes a plainList
. Auto-value-gson constructs theFoo
instance using its builder, and the deserializedList
works with thesetBars
method. That method callsImmutableList.copyOf(List)
so we do get the rightImmutableList
type. But the additional checks in this PR don't let us get that far.As I say, code like that only works accidentally. My plan is to add a
GuavaCollectionsTypeAdapterFactory
toextras
and update the internal code to use that. I already found several customImmutableListTypeAdapterFactory
andImmutableMapTypeAdapterFactory
classes which could be switched to use this.Some of the other failures were just plain using the wrong types, which again happened to work through erasure and the like.
So in short, I think this change is OK, but it may require some client code to be updated. Most of the updates should be straightforward and will lead to client code that is more correct.
I think it will be a while before we can complete this because of all the adjustments that will need to be made.