Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Permit CLDR rather than ISO 4217 for CurrencyDigits? #134

Closed
littledan opened this issue Mar 10, 2017 · 23 comments · Fixed by #922
Closed

Permit CLDR rather than ISO 4217 for CurrencyDigits? #134

littledan opened this issue Mar 10, 2017 · 23 comments · Fixed by #922
Assignees
Labels
c: numbers Component: numbers, currency, units s: help wanted Status: help wanted; needs proposal champion Small Smaller change solvable in a Pull Request web reality
Milestone

Comments

@littledan
Copy link
Member

CurrencyDigits is an abstract operation which normatively references ISO 4217 to get the default number of digits to display for a currency. That data can be found on ISO 4217's website in this XML file.

CLDR has its own notion of how many digits should be used for currency display by default. Here is the data for CLDR 30.

These charts actually differ in a number of places. For example, afghani (AFN) is listed as having 2 decimal digits in ISO 4217, but 0 in CLDR 30. According to Wikipedia, the new afghani (AFN) has never had subdivisions issued, though it is nominally divided into 100 smaller units.

I'm not sure what accounts for this difference, whether ISO 4217 is going for more legalistic definitions and CLDR is going for what's linguistically relevant, or whether one has fresher data than the other, or something else. I'd be interested if anyone has more context here.

V8 has been using CLDR data for CurrencyDigits, whereas SpiderMonkey (and now JSC) has been using ISO 4217 data. I wrote patches to switch SpiderMonkey and JSC to switch to CLDR, not realizing, until it was caught in SpiderMonkey's test automation (thanks @syg!), that this was actually a semantic change, and introduced a spec violation caught by a test262 test.

Apparently, either source is web-compatible, but I don't know which one is better. Does anyone have more background here?

cc @jungshik

@NorbertLindenberg
Copy link

The ISO 4217 definition of the decimal digits for currencies is indeed legalistic and sometimes detached from reality. I contacted the maintenance agency in 2013 to ask about the Indonesian rupiah, which was listed as having two decimal digits. In reality, the smallest coin in Indonesia is 50 rupiah, which is worth about half a US cent, and a 1000:1 redenomination of the rupiah has been considered for years. The maintenance agency contacted the Central Bank of Indonesia, which replied that the minor unit was still used in accounting and therefore should remain. And so it remained.

@littledan
Copy link
Member Author

@NorbertLindenberg Interesting details here. This sort of logic seems reasonable for ISO 4217 itself, to be a place where central banks may give details of official denominations. At the same time, I can see why CLDR would diverge.

Which do you think is more salient for web platform users who want to display numbers appropriately in a particular currency? Would you object to switching to an unspecified data source, with a note which makes a CLDR optional cross-reference suggestion, as many other parts of the ECMA 402 spec are?

@NorbertLindenberg
Copy link

In general, the standard should prescribe the behavior of ECMAScript API as closely as possible. The Internationalization API got some dispensation in that regard for the reasons outlined in section 4.3, but since there is a standard source for the decimal digits for currencies, even if it's not ideal, switching to an unspecified data source would be heading in the wrong direction. Switching to CLDR as a normative data source, on the other hand, might be acceptable, although somebody with more experience in dealing with ISO should confirm whether it indeed is. It might help if someone from the CLDR team could describe or point to their rules for determining currency information.

I think applications that use the API for business or accounting purposes should specify the decimal digits themselves anyway. Relying on the API implementation to provide this information is more acceptable for applications that provide prices in the user's currency as estimates only, e.g., when displaying the rates for hotels in foreign countries that charge in their local currency.

@littledan
Copy link
Member Author

@allenwb @ecmageneva Do you have any understanding about what it would mean for ISO to normatively reference CLDR from a TC39 spec? Or who to ask?

@bterlson What would you think of normatively referencing to CLDR here?

@NorbertLindenberg I can see the high-level goal of making things as tied down as possible, but the ECMA 402 specifcation defers to unspecified information sources for all of its core functionality. It only includes a couple normative references at the edges which go beyond Unicode and BCP 47 (here, the timezone database). Given that practical implementations will be tying into an existing internationalization library with its stance on these issues (whether it's ICU or Windows Unicode libraries or something else), it seems reasonable to me to say, "you can use your platform's best understanding of what's linguistically appropriate here", as is done elsewhere; this would be a step up from normatively referencing something that we know to be less accurate compared to data that's already in the platform.

@caridy
Copy link
Contributor

caridy commented Mar 17, 2017

@bterlson What would you think of normatively referencing to CLDR here?

This is the point of contention here. We will have to wait for Microsoft's formal response about this, and hopefully @bterlson can help.

Somebody with more experience in dealing with ISO should confirm whether it indeed is. It might help if someone from the CLDR team could describe or point to their rules for determining currency information.

Absolutely, let's loop me more folks that might help to clarify this. /cc @srl295 @JCEmmons

@JCEmmons
Copy link

I think @NorbertLindenberg was absolutely correct when describing ISO 4217 definition as somewhat legalistic and often outdated. In CLDR land, we try to go for "every day usage" ( i.e. what would you want to see if you were buying something online ). Often I will use things like Wikipedia as a reference. I will look to see what the lowest value coin in circulation is. I have had numerous times over the years where I have tried to get ISO to change. Sometimes the only place where you would see fractional digits would be in banking transactions. Now in CLDR we actually have added "cashDigits" and "cashRounding" to help handle some of the edge case currencies like TWD, where street usage is definitely 0 decimal places, but 2 decimal places in the banks.

Kind of like some of the oddities you see here in the U.S. When you go to the gas station, you will almost always see the price as $x.xx9 per gallon. Why the extra 9/10 cent I will never know.....

@mattjohnsonpint
Copy link

Why the extra 9/10 cent I will never know.....

I've always thought that was about taxes, but the Internet says it's a marketing ploy.

@littledan
Copy link
Member Author

OK, so if the CLDR data is considered better than the ISO 4217 data, would it be acceptable to change the specification to allow using it? I'm having trouble making sense of @NorbertLindenberg 's comments--I could make a very well-specified but very wrong database for just about anything in Intl, but most things are left to be provided by the implementation to permit greater accuracy.

@littledan
Copy link
Member Author

FWIW someone actually filed a bug against Chrome for violating the spec and using CLDR data. However, it appears that the real issue here was its differing output from Safari and Firefox.

@annevk
Copy link
Member

annevk commented Sep 12, 2017

FWIW, the HTML Standard also uses CLDR for one thing. Seems much less problematic than ICU at least.

@littledan
Copy link
Member Author

Would anyone be interested in compiling a case of differences between CLDR and ISO 4217, as @apaprocki suggested on Twitter ?

@srl295
Copy link
Member

srl295 commented Feb 23, 2018

@annevk

FWIW, the HTML Standard also uses CLDR for one thing. Seems much less problematic than ICU at least.

What does this mean?

@littledan I responded on Twitter… Maybe someone would contribute a list that could be a part of CLDR noting deviation from 4217, or maybe a tool to perform the diff?

@apaprocki
Copy link

@NorbertLindenberg The Bloomberg Terminal is in agreement with CLDR over the Indonesian rupiah; it is quoted with 0 decimal places in the system

@littledan Interestingly, though, the Afghani is quoted in (and uses) 4 decimal places. In this case, it appears an Afghan bank rate provider does only quote out to 2 decimal places, but other FX venues go out to 4 decimal places, so the composite spot uses 4. I would need to ask around to dig deeper on this particular currency. At first glance, though, I'd venture the CLDR value of 0 has to be wrong.

@apaprocki
Copy link

@littledan For the AFN case, it was added to the supplementalData.xml file way back in 2008 with no real description of why all the values in the diff should be 0 or what methodology was used to fact-check / reality-check it, and it hasn't changed since.

@jungshik
Copy link

@apaprocki would you file a ticket against CLDR about AFN ?

@jungshik
Copy link

https://bugs.chromium.org/p/v8/issues/detail?id=7472 is a v8 bug for failing to pass
test262/intl402/NumberFormat/currency-digits .

@jungshik
Copy link

@littledan
Copy link
Member Author

We discussed this issue on a tangent in the November 2018 Intl meeting. At the time, there seemed to be interest in moving towards a specification which permits tailoring. Let's investigate details in a PR.

@sffc sffc added help wanted c: numbers Component: numbers, currency, units s: help wanted Status: help wanted; needs proposal champion and removed s: help wanted labels Mar 19, 2019
@littledan
Copy link
Member Author

@spectranaut Would you be interested in taking this on as a follow-up to the case tailoring issue?

@sffc sffc added this to the ES 2021 milestone Mar 20, 2020
@sffc sffc added the Small Smaller change solvable in a Pull Request label Mar 20, 2020
@sffc sffc modified the milestones: ES 2021, ES 2022 Mar 22, 2021
@sffc sffc modified the milestones: ES 2022, ES 2023 Jun 1, 2022
@felixfbecker
Copy link

I just stumbled across this going down the rabbit hole of why some currencies are formatted differently in Chrome compared to other browsers. One part that is especially confusing here is that the documentation on MDN explicitly states that the default number of fractional digits is taken from ISO 4217, when apparently that is actually not mandated by the spec and not the case in Chrome (where CLDR is used).

@srl295
Copy link
Member

srl295 commented Jun 29, 2023

CLDR carefully takes in iso 4217 updates (I've done several of them recently) but as with anything can adjust for reality "on the ground". Something I'm not sure is captured in this issue is that CLDR has separate info for cash vs non cash transactions. Perhaps the non cash might still differ from 4217

@felixfbecker
Copy link

felixfbecker commented Jun 29, 2023

Interesting thought. The non cash values still differ from ISO 4217 – for example IQD (Iraqi Dinar) is listed as 3 minor units in ISO 4217 [1] but listed as 0 minor units in CLDR [2], with extra no value listed in the cash units column.

CleanShot 2023-06-29 at 10 38 52@2x CleanShot 2023-06-29 at 10 38 40@2x

[1] https://www.six-group.com/dam/download/financial-information/data-center/iso-currrency/lists/list-one.xml search for "IQD"
[2] https://github.com/unicode-org/cldr/blob/release-43-1/common/supplemental/supplementalData.xml#L36 or in the release charts supplemental/detailed_territory_currency_information.html#IQD

@sffc
Copy link
Contributor

sffc commented Sep 18, 2023

CC @ben-allen to put together a PR for changing to CLDR as the normative reference so that we can gather feedback.

ben-allen added a commit to ben-allen/ecma402 that referenced this issue Sep 10, 2024
This PR allows for locale data other than ISO 4217 for determining
the number of fractional digits to be used when formatting currency
values.

In some applications ISO 4217 is most appropriate, in some CLDR data,
and in some neither is best. See tc39#921 (comment)

Resolves tc39#134

This PR makes the `CurrencyDigits` AO implementation-defined.
ben-allen added a commit to ben-allen/ecma402 that referenced this issue Sep 12, 2024
This PR allows for locale data other than ISO 4217 for determining
the number of fractional digits to be used when formatting currency
values.

In some applications ISO 4217 is most appropriate, in some CLDR data,
and in some neither is best. See tc39#921 (comment)

Resolves tc39#134

This PR makes the `CurrencyDigits` AO implementation-defined.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: numbers Component: numbers, currency, units s: help wanted Status: help wanted; needs proposal champion Small Smaller change solvable in a Pull Request web reality
Projects
None yet