-
Notifications
You must be signed in to change notification settings - Fork 627
Add identifier unicode support in Mysql, Postgres and Redshift #1933
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @etgarperets, could you also take a look at the ci failure (and also to merge in latest from main as there are currently some conflicts on the branch)
tests/sqlparser_common.rs
Outdated
let unicode_sql = r#"SELECT phoneǤЖשचᎯ⻩☯♜🦄⚛🀄ᚠ⌛🌀 tbl FROM customers"#; | ||
let dialects_supporting_unicode = TestedDialects::new(vec![Box::new(MySqlDialect {}), Box::new(RedshiftSqlDialect {}), Box::new(PostgreSqlDialect {})]); | ||
let _ = dialects_supporting_unicode.parse_sql_statements(unicode_sql).unwrap(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let unicode_sql = r#"SELECT phoneǤЖשचᎯ⻩☯♜🦄⚛🀄ᚠ⌛🌀 tbl FROM customers"#; | |
let dialects_supporting_unicode = TestedDialects::new(vec![Box::new(MySqlDialect {}), Box::new(RedshiftSqlDialect {}), Box::new(PostgreSqlDialect {})]); | |
let _ = dialects_supporting_unicode.parse_sql_statements(unicode_sql).unwrap(); | |
let sql = r#"SELECT phoneǤЖשचᎯ⻩☯♜🦄⚛🀄ᚠ⌛🌀 tbl FROM customers"#; | |
let dialects = TestedDialects::new(vec![Box::new(MySqlDialect {}), Box::new(RedshiftSqlDialect {}), Box::new(PostgreSqlDialect {})]); | |
let _ = dialects.verified_stmt(unicode_sql); |
tests/sqlparser_common.rs
Outdated
@@ -15895,3 +15895,11 @@ fn parse_create_procedure_with_parameter_modes() { | |||
_ => unreachable!(), | |||
} | |||
} | |||
|
|||
#[test] | |||
fn test_unicode_support() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fn test_unicode_support() { | |
fn test_identifier_unicode_support() { |
src/dialect/mysql.rs
Outdated
@@ -51,7 +51,7 @@ impl Dialect for MySqlDialect { | |||
} | |||
|
|||
fn is_identifier_part(&self, ch: char) -> bool { | |||
self.is_identifier_start(ch) || ch.is_ascii_digit() | |||
self.is_identifier_start(ch) || ch.is_ascii_digit() || !ch.is_ascii() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add a comment to these highlighting that they implement unicode character support?
…o added a test for that
bdd6f57
to
c03a7d7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Thanks @etgarperets!
cc @alamb
…o added a test for that