Fix the Serialization/Deserialization issue with $ prefix columns

[Alekhya-Polavarapu]

Why make this change?

Serialization and deserialization of metadata currently fail when column names are prefixed with the $ symbol.

Root cause

This issue occurs because we’ve enabled the ReferenceHandler flag in our System.Text.Json serialization settings. When this flag is active, the serializer treats $ as a reserved character used for special metadata (e.g., $id, $ref). As a result, any property name starting with $ is interpreted as metadata and cannot be deserialized properly.

What is this change?

This update introduces custom logic in the converter’s Write and Read methods to handle $-prefixed column names safely.

• During serialization, columns beginning with $ are escaped as "_$".

• During deserialization, this transformation is reversed to restore the original property names.

How was this tested

• Unit tests

[Copilot]

Pull Request Overview

This PR adds support for escaping dollar sign ($) prefixes in column names during JSON serialization/deserialization. This is necessary because JSON property names starting with $ have special meaning in some contexts (like MongoDB's JSON query language) and can cause serialization issues.

• Adds escape/unescape logic to DatabaseObjectConverter that converts $ to _$ during serialization and back during deserialization
• Updates InitializeObjects method to optionally generate column names with dollar sign prefixes
• Adds three new test methods to validate serialization/deserialization of database objects with dollar-prefixed column names

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 9 comments.

File	Description
src/Core/Services/MetadataProviders/Converters/DatabaseObjectConverter.cs	Implements escaping/unescaping logic for dollar sign prefixed column names in SourceDefinition properties
src/Service.Tests/UnitTests/SerializationDeserializationTests.cs	Adds test coverage for dollar sign handling and updates initialization method to support parameterized column name generation

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Alekhya-Polavarapu]

Can we confidently escape it to "_$", may be there might be columns that are genuinely created with _$.

[souvikghosh04]

$ or _$ are allowed in some databases but used in different ways. for e.g., in PostgreSQL, the column should be used in within double-quotes. In MS SQL, its allowed but in box brackets I suppose. MySQL allows too. its better to handle these edge cases and also test it with different databases.

[JerryNixon]

We should consider if this is an edge case we care about.

[Copilot]

Pull Request Overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Corrected spelling of 'deserilization' to 'deserialization'.

[Copilot]

The result of Equals() method call is not being asserted. This should be Assert.IsTrue(deserializedDatabaseTable.Equals(_databaseTable)) or similar to validate equality.