Workaround for limitations of generics

[peturingi]

I encountered a limitation when using generic type parameters (ETypeParameter) with EAttribute in Ecore models.
The issue manifests when class A<T> contains a reference to class B<T>, where B has an EAttribute of type T.
In that case, the EAttribute's eType resolves to EJavaObject at runtime.
During serialisation and deserialisation (both XML and binary formats), EMF cannot determine the effective EType, causing the value to be serialised as bytes as if it was an instance of java.lang.Object.

Simple Reproduction Case

Given this Ecore model:

  <eClassifiers xsi:type="ecore:EClass" name="Document">
    <eStructuralFeatures xsi:type="ecore:EReference" name="a" containment="true">
      <eGenericType eClassifier="#//A">
        <eTypeArguments eClassifier="#//Hex"/>
      </eGenericType>
    </eStructuralFeatures>
  </eClassifiers>
  
  <eClassifiers xsi:type="ecore:EClass" name="A">
    <eTypeParameters name="T"/>
    <eStructuralFeatures xsi:type="ecore:EReference" name="b" containment="true">
      <eGenericType eClassifier="#//B">
        <eTypeArguments eTypeParameter="#//A/T"/>
      </eGenericType>
    </eStructuralFeatures>
  </eClassifiers>
  
  <eClassifiers xsi:type="ecore:EClass" name="B">
    <eTypeParameters name="T"/>
    <eStructuralFeatures xsi:type="ecore:EAttribute" name="value">
      <eGenericType eTypeParameter="#//B/T"/>
    </eStructuralFeatures>
  </eClassifiers>
  
  <eClassifiers xsi:type="ecore:EDataType" name="Hex" instanceClassName="java.lang.Integer"/>

When serialising Document.a.b.value with value 42:

• Expected: <value>0x2A</value> (using Hex's string conversion from factory)
• Actual: <value>ACED0005737200116A6176612E6C616E672E496E746567657212E2A0A4F781873802000149000576616C7565787200106A6176612E6C616E672E4E756D62657286AC951D0B94E08B02000078700000002A</value> (using EJavaObject/Integer default)

Analysis

1. The object hasn't been added to the containment tree yet during deserialisation
2. Type resolution requires walking up the containment hierarchy to find the type arguments from the containing reference
3. The generic type information is only available through eGenericType, not through the standard eType

This results in values being stored/loaded as strings or incorrect types, breaking roundtrip serialisation.

Root Cause

The issue stems from the temporal ordering of EMF's deserialisation process. During XML loading, setValue is called before the object is added to the containment tree, preventing proper type resolution:

┌──────────┐          ┌────────────┐      ┌────────────┐      ┌──────────┐      ┌─────────────┐
│XMLHandler│          │ XMLHelper  │      │  EObject   │      │EAttribute│      │EcorePackage │
└────┬─────┘          └─────┬──────┘      └─────┬──────┘      └────┬─────┘      └──────┬──────┘
     │                      │                   │                   │                   │
     │ handleAttribute()    │                   │                   │                   │
     │──────────────────────>                   │                   │                   │
     │   value="0x2A"       │                   │                   │                   │
     │                      │                   │                   │                   │
     │              setValue(object, feature,   │                   │                   │
     │                       value, -1)         │                   │                   │
     │                      │──────────────────>│                   │                   │
     │                      │                   │                   │                   │
     │                      │                   │  eContainer()     │                   │
     │                      │                   │<─ ─ ─ ─ ─ ─ ─ ─ ─ │                   │
     │                      │                   │  returns null!    │                   │
     │                      │                   │                   │                   │
     │                      │           getEType()                  │                   │
     │                      │                   │──────────────────>│                   │
     │                      │                   │   EJavaObject     │                   │
     │                      │                   │<──────────────────│                   │
     │                      │                   │                   │                   │
     │                      │          ╔════════════════════════════════════════════╗   │
     │                      │          ║ NOTE: Cannot resolve eGenericType because  ║   │
     │                      │          ║ object not yet in containment tree!        ║   │
     │                      │          ║ Falls back to declared eType=EJavaObject   ║   │
     │                      │          ╚════════════════════════════════════════════╝   │
     │                      │                   │                   │                   │
     │                      │  EcoreUtil.createFromString(          │                   │
     │                      │           EJavaObject, "0x2A")        │                   │
     │                      │───────────────────────────────────────────────────────────>
     │                      │                   │                   │                   │
     │                      │           ╔═══════════════════════════════════════════╗   │
     │                      │           ║ PROBLEM: Uses EcorePackage factory for    ║   │
     │                      │           ║ EJavaObject instead of Hex factory!       ║   │
     │                      │           ║ Result: NumberFormatException or wrong    ║   │
     │                      │           ║ conversion (treats as Integer not Hex)    ║   │
     │                      │           ╚═══════════════════════════════════════════╝   │
     │                      │                   │                   │     Integer(42)   │
     │                      │<───────────────────────────────────────────────────────────
     │                      │                   │                   │                   │
     │                      │          eSet(feature, value)         │                   │
     │                      │                   │──────────────────>│                   │
     │                      │                   │  wrong type!      │                   │
     │<─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─                    │                   │                   │
     │                      │                   │                   │                   │
     │  ╔══════════════════════════════════════════════════════════════════════════╗    │
     │  ║ Object added to containment tree AFTER setValue completes                ║    │
     │  ║ Now eContainer() would return parent, but too late!                      ║    │
     │  ║ Type information (A<Hex>.reference : B<Hex>) is now available but unused ║    │
     │  ╚══════════════════════════════════════════════════════════════════════════╝    │
     │                      │                   │                   │                   │

When EAttribute has eGenericType with an eTypeParameter, the resolution chain is:

1. Object's eClass() defines eTypeParameters (e.g., <T extends EDataType>)
2. Object's container's containment EReference has eGenericType with eTypeArguments (e.g., Container<Integer>)
3. The type argument at the parameter's index gives us the effective EType (Integer)

But during deserialisation, the object isn't in the containment tree yet, so the resolution fails.

Workaround

A post-processing approach that resolves generic types after the containment tree is complete:

Deserialisation (XML/Binary)
 - 1. Standard EMF loading (values stored as strings/objects)
 - 2. Containment tree established
 - 3. Post-process: Walk tree, resolve generics, convert

Serialisation (XML)
 - 1. Pre-process: Resolve generic type during save
 - 2. Use effective eType for conversion

Implementation

Four extension classes:

1. XML Deserialisation - Store Strings Temporarily

/**
 * Stores string values for unresolved generic attributes during loading.
 */
public class FixedXMLHelperImpl extends XMLHelperImpl {

    public FixedXMLHelperImpl(XMLResource resource) {
        super(resource);
    }

    @Override
    public void setValue(EObject object, EStructuralFeature feature, Object value, int position) {
        /* Pre: object <> null
               and feature <> null */
        // Handle EJavaObject attributes with generic types - store string as-is for post-processing
        if (feature instanceof EAttribute && value instanceof String) {
            EClassifier eType = feature.getEType();
            if ("EJavaObject".equals(eType.getName()) && feature.getEGenericType() != null) {
                if (feature.isMany()) {
                    @SuppressWarnings("unchecked")
                    java.util.List<Object> list = (java.util.List<Object>) object.eGet(feature);
                    if (position == -1) {
                        list.add(value);
                    } else {
                        list.add(position, value);
                    }
                } else {
                    object.eSet(feature, value);
                }
                return;
            }
        }
        super.setValue(object, feature, value, position);
        /* Post: object.eIsSet(feature)
                or value = null */
    }
}

2. XML Deserialisation - Post-Process After Loading

/**
 * Post-processes loaded models to resolve generic type parameters.
 */
public class FixedXMLLoadImpl extends XMLLoadImpl {

    public FixedXMLLoadImpl(XMLHelper helper) {
        super(helper);
    }

    @Override
    public void load(XMLResource resource, java.io.InputStream inputStream, java.util.Map<?, ?> options) throws java.io.IOException {
        /* Pre: resource <> null
               and inputStream <> null */
        // O(n*a*d) where n = objects, a = attributes, d = containment depth
        super.load(resource, inputStream, options);

        for (EObject root : resource.getContents()) {
            fixGenericTypes(root);
        }
        /* Post: resource.getContents()->forAll(obj |
                    obj.eClass().eAllAttributes->forAll(attr |
                        attr.eGenericType <> null
                        and attr.eGenericType.eTypeParameter <> null
                        implies obj.eIsSet(attr))) */
    }

    @Override
    public void load(XMLResource resource, org.xml.sax.InputSource inputSource, java.util.Map<?, ?> options) throws java.io.IOException {
        /* Pre: resource <> null
               and inputSource <> null */
        // O(n*a*d) where n = objects, a = attributes, d = containment depth
        super.load(resource, inputSource, options);

        for (EObject root : resource.getContents()) {
            fixGenericTypes(root);
        }
        /* Post: resource.getContents()->forAll(obj |
                    obj.eClass().eAllAttributes->forAll(attr |
                        attr.eGenericType <> null
                        and attr.eGenericType.eTypeParameter <> null
                        implies obj.eIsSet(attr))) */
    }

    private void fixGenericTypes(final EObject object) {
        /* Pre: object <> null
               and object.eClass() <> null */
        // O(n) tree traversal where n = total objects in subtree
        final EClass eClass = object.eClass();

        for (final EAttribute attribute : eClass.getEAllAttributes()) {
            if (attribute.getEGenericType() != null && attribute.getEGenericType().getETypeParameter() != null) {
                fixAttributeValue(object, attribute);
            }
        }

        for (final EObject child : object.eContents()) {
            fixGenericTypes(child);
        }
        /* Post: object.eClass().eAllAttributes->forAll(attr |
                    attr.eGenericType <> null
                    and attr.eGenericType.eTypeParameter <> null
                    implies object.eIsSet(attr))
               and object.eContents()->forAll(child |
                    child.eClass().eAllAttributes->forAll(attr |
                        attr.eGenericType <> null
                        and attr.eGenericType.eTypeParameter <> null
                        implies child.eIsSet(attr))) */
    }

    private void fixAttributeValue(final EObject object, final EAttribute attribute) {
        /* Pre: object <> null
               and attribute <> null
               and attribute.eGenericType <> null
               and attribute.eGenericType.eTypeParameter <> null */
        // O(d) for resolution + O(1) for conversion where d = containment depth
        Object currentValue = object.eGet(attribute);
        if (currentValue == null) {
            return;
        }

        EDataType effectiveType = getEffectiveEType(object, attribute);
        EDataType declaredType = (EDataType) attribute.getEType();

        if (effectiveType != declaredType && !"EJavaObject".equals(effectiveType.getName())) {
            EFactory factory = effectiveType.getEPackage().getEFactoryInstance();

            if (attribute.isMany()) {
                @SuppressWarnings("unchecked")
                java.util.List<Object> list = (java.util.List<Object>) currentValue;
                for (int i = 0; i < list.size(); i++) {
                    Object item = list.get(i);
                    if (item != null) {
                        String stringValue;
                        if (item instanceof String) {
                            stringValue = (String) item;
                        } else {
                            stringValue = EcoreUtil.convertToString(declaredType, item);
                        }
                        Object newValue = factory.createFromString(effectiveType, stringValue);
                        list.set(i, newValue);
                    }
                }
            } else {
                String stringValue;
                if (currentValue instanceof String) {
                    stringValue = (String) currentValue;
                } else {
                    stringValue = EcoreUtil.convertToString(declaredType, currentValue);
                }
                Object newValue = factory.createFromString(effectiveType, stringValue);
                object.eSet(attribute, newValue);
            }
        }
        /* Post: object.eIsSet(attribute)
                or object.eGet(attribute) = null */
    }

    private EDataType getEffectiveEType(final EObject eObject, final EStructuralFeature f) {
        /* Pre: eObject <> null
               and f <> null */
        // O(d) where d = containment depth
        final EGenericType genericType = f.getEGenericType();
        if (genericType != null) {
            if (resolveGenericType(eObject, genericType) instanceof EDataType resolved) {
                return resolved;
            }
        }
        return (EDataType) f.getEType();
        /* Post: result <> null
                and result.oclIsKindOf(EDataType) */
    }

    private EClassifier resolveGenericType(final EObject context, final EGenericType genericType) {
        /* Pre: context <> null
               and genericType <> null */
        // O(d) where d = containment depth (recursive)
        final ETypeParameter typeParam = genericType.getETypeParameter();
        if (typeParam != null) {
            final EClass definingClass = context.eClass();
            final int paramIndex = definingClass.getETypeParameters().indexOf(typeParam);
            if (paramIndex >= 0) {
                final EObject container = context.eContainer();
                if (container != null) {
                    final EReference containmentFeature = context.eContainmentFeature();
                    if (containmentFeature != null) {
                        EGenericType containerGenericType = containmentFeature.getEGenericType();
                        if (containerGenericType != null) {
                            if (containerGenericType.getETypeParameter() != null) {
                                containerGenericType = findGenericTypeInContainer(container, containerGenericType);
                            }
                            if (containerGenericType != null
                                    && !containerGenericType.getETypeArguments().isEmpty()
                                    && paramIndex < containerGenericType.getETypeArguments().size()) {
                                final EGenericType typeArgument = containerGenericType.getETypeArguments().get(paramIndex);
                                return resolveGenericType(container, typeArgument);
                            }
                        }
                    }
                }
            }

            // Liskov Substitution: treat "T extends Bound" as Bound when actual type cannot be determined
            if (typeParam.getEBounds() != null && !typeParam.getEBounds().isEmpty()) {
                final EGenericType bound = typeParam.getEBounds().getFirst();
                if (bound != null) {
                    final EClassifier boundClassifier = bound.getEClassifier();
                    if (boundClassifier != null) {
                        return boundClassifier;
                    }
                    if (bound.getETypeParameter() != null) {
                        return resolveGenericType(context, bound);
                    }
                }
            }
            return genericType.getEClassifier();
        }
        return genericType.getEClassifier();
        /* Post: result = null
                or result.oclIsKindOf(EClassifier) */
    }

    private EGenericType findGenericTypeInContainer(final EObject context, final EGenericType genericType) {
        /* Pre: context <> null
               and genericType <> null */
        // O(d) where d = containment depth (recursive)
        final ETypeParameter typeParam = genericType.getETypeParameter();
        if (typeParam == null) {
            return genericType;
        }

        final EClass definingClass = context.eClass();
        final int paramIndex = definingClass.getETypeParameters().indexOf(typeParam);
        if (paramIndex < 0) {
            return null;
        }

        final EObject container = context.eContainer();
        if (container == null) {
            return null;
        }

        final EReference containmentFeature = context.eContainmentFeature();
        if (containmentFeature == null) {
            return null;
        }

        EGenericType containerGenericType = containmentFeature.getEGenericType();
        if (containerGenericType == null) {
            return null;
        }

        if (containerGenericType.getETypeParameter() != null) {
            containerGenericType = findGenericTypeInContainer(container, containerGenericType);
            if (containerGenericType == null) {
                return null;
            }
        }

        if (containerGenericType.getETypeArguments().isEmpty()
                || paramIndex >= containerGenericType.getETypeArguments().size()) {
            return null;
        }

        return containerGenericType.getETypeArguments().get(paramIndex);
        /* Post: result = null
                or (result.oclIsKindOf(EGenericType)
                    and (result.eTypeParameter = null
                         implies not result.eTypeArguments->isEmpty())) */
    }
}

3. XML Serialisation - Resolve During Save

/**
 * Resolves generic type parameters during XML serialisation.
 */
public class FixedXMLSaveImpl extends org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl {

    protected static final String XSI_NIL = XMLResource.XSI_NS + ":" + XMLResource.NIL;

    public FixedXMLSaveImpl(final XMLHelper helper) {
        super(helper);
    }

    public FixedXMLSaveImpl(final Map<?, ?> options, final XMLHelper helper, final String encoding) {
        super(options, helper, encoding);
    }

    public FixedXMLSaveImpl(final Map<?, ?> options, final XMLHelper helper, final String encoding, final String xmlVersion) {
        super(options, helper, encoding, xmlVersion);
    }

    @Override
    protected void saveDataTypeSingle(final EObject o, final EStructuralFeature f) {
        /* Pre: o <> null
               and f <> null */
        // O(d) where d = containment depth for type resolution
        if (f.getEGenericType() == null) {
            super.saveDataTypeSingle(o, f);
            return;
        }

        final Object value = helper.getValue(o, f);
        if (value == null) {
            return;
        }

        final String svalue = convertValue(o, f, value);
        if (svalue != null) {
            writeAttribute(f, svalue, o, value);
        }
        /* Post: true */
    }

    private String convertValue(final EObject o, final EStructuralFeature f, final Object value) {
        /* Pre: o <> null
               and f <> null
               and value <> null */
        // O(d) where d = containment depth
        final EDataType effectiveType = getEffectiveEType(o, f);
        final EFactory factory = effectiveType.getEPackage().getEFactoryInstance();
        String svalue = helper.convertToString(factory, effectiveType, value);
        return escape != null ? escape.convert(svalue) : svalue;
        /* Post: result <> null */
    }

    // ... (remaining helper methods and type resolution logic similar to LoadImpl)

    protected EDataType getEffectiveEType(final EObject eObject, final EStructuralFeature f) {
        /* Pre: eObject <> null
               and f <> null */
        // O(d) where d = containment depth
        final EGenericType genericType = f.getEGenericType();
        if (genericType != null) {
            if (resolveGenericType(eObject, genericType) instanceof  EDataType resolved) {
                return resolved;
            }
        }
        return (EDataType) f.getEType();
        /* Post: result <> null
                and result.oclIsKindOf(EDataType) */
    }

    // ... (resolveGenericType and findGenericTypeInContainer same as LoadImpl)
}

4. Binary Format Support

/**
 * Post-processes binary-loaded models to resolve generic type parameters.
 */
public class FixedBinaryResourceImpl extends BinaryResourceImpl {

    public FixedBinaryResourceImpl() {
        super();
    }

    public FixedBinaryResourceImpl(URI uri) {
        super(uri);
    }

    @Override
    protected void doLoad(final InputStream inputStream, final Map<?, ?> options) throws IOException {
        /* Pre: inputStream <> null */
        // O(n*a*d) where n = objects, a = attributes, d = containment depth
        super.doLoad(inputStream, options);

        for (final EObject root : getContents()) {
            fixGenericTypes(root);
        }
        /* Post: getContents()->forAll(obj |
                    obj.eClass().eAllAttributes->forAll(attr |
                        attr.eGenericType <> null
                        and attr.eGenericType.eTypeParameter <> null
                        implies obj.eIsSet(attr))) */
    }

    // ... (same post-processing as XML LoadImpl)
}

Usage

// For XML:
ResourceFactoryImpl resourceFactory = new XMIResourceFactoryImpl() {
    @Override
    public Resource createResource(URI uri) {
        return new XMIResourceImpl(uri) {
            @Override
            protected XMLLoad createXMLLoad() {
                return new FixedXMLLoadImpl(createXMLHelper());
            }
            @Override
            protected XMLSave createXMLSave() {
                return new FixedXMLSaveImpl(createXMLHelper());
            }
            @Override
            protected XMLHelper createXMLHelper() {
                return new FixedXMLHelperImpl(this);
            }
        };
    }
};

// For Binary:
ResourceFactoryImpl binaryFactory = new FixedBinaryResourceFactoryImpl();

Rationale

Why Post-Processing?

Decided on post-processing over modifying EMF's core serialisation because:

1. Non-invasive and works with existing EMF infrastructure without requiring core changes
2. Doesn't break existing models/code (backwards compatible)
3. All logic is contained in a few extension classes (simplicity)
4. Performance: O(nad), where n = objects, a = attributes and d = containment depth, complexity is somewhat acceptable for models with generic attributes

Complexity

• Tree traversal: O(n) where n = total objects
• Per attribute: O(d) type resolution where d = containment depth
• Overall: O(n*a*d) where a = attributes with generic types per class; n = total objects; d = containment depth
• In practice: Most classes have 0-2 generic attributes, containment is shallow, so performance is reasonable for typical models

Design Decisions

1. Leverages existing EFactory.createFromString() for values coming in a strings during XML Load
2. Binary uses same post-processing
3. When type can't be resolved, fall back to T extends Bound (Liskov Substitution Principle)
4. Stateless implementation: Safe for concurrent use, no side effects

Limitations

• Requires containment tree to be complete (works for standard load, may not work for incremental loading scenarios)
• Only handles EAttribute with EDataType generic parameters (not EReference with generic EClass)
• Assumes EDataType has instanceClassName defined for conversion
• Post-processing overhead proportional to tree size and generic attribute count
• No proxy resolution handling in current implementation
• Does not handle complex nested generics (e.g., EList<T>)
• Does not work with FeatureMap or mixed content scenarios

Open Questions

Through my current solution is an external workaround, I am wondering:

1. Would this type of functionality be valuable in EMF core, or is the workaround preferable?

2. Implementation Location: If this were to be included, where would it best fit?

   • In the default XMI/Binary resource implementations?
   • EcoreTools?
   • A separate plugin?

3. Are there more straight forward ways, which I have missed, to handle this? I thought about:

   • Modifying XMLHandler to delay type conversion (too invasive).
   • Custom templates / genmodel-generation (maintenance burden).
   • Altering the EFactory/EClass in org.eclipse.emf.ecore (too invasive).

4. If deemed a valuable contribution to EMF core, should this handle more than just EAttribute with EDataType parameters? I'm thinking:

   • Generic EReference types,
   • Nested isMany := true´ generics such as EList`?
   • Something else I am missing?

I would happy to refactor this implementation to match EMF's standards if there's interest in incorporating it.
I primarily wanted to share my approach and get feedback from the community.

Thank you @merks for your time and for maintaining such a powerful framework!

[merks]

This looks AI generated. It's beyond my capacity to deal with high volume AI content.

I think there will always be limitations if the type must be inferred from some "type binding" information of some container arbitrarily higher up in the containment chain. Even there the type's details can depend recursively on type parameters that turn must be resolved via some known "type binding" information even higher in the containment chain. This looks to me like a can of worms where the worms will never stay in the can. In any case, it must always be possible to serialize and deserialize B in isolation of the container, e.g., with cross resource containment references, where the container might to be loaded from a different resource separately from B an hence only a proxy is available, or just in general that any instance, in isolation, can be serialized or deserialized.