Make Parsing error more user friendly

[rbillott]

Hi there.

First of all, thank you all for the good work. CsvHelper is indeed very helpful for what I need to do: ie CSV parsing.

Helpful but, my users are very unhappy with the errors my app sometime displays when the CSV is not the expected one.
I'm using the Exception message which definitely contains what we need to investigate but not suitable for the display to end user; End-user need something minimalist and meaningful, easy to understand and instead we get a lot of confusing technical information that requires expert eyes.

I understand that what is currently printed is useful for you guys developer in order to debug, but again not suitable for non IT. So I thing a general solution would be to provide to 2 versions of the error. Then the question is how to get those 2 versions.. can it be by config, or Message vs ToString() or a new property FriendlyMessage ?? (that's up to you)

Below are 2 simple examples that should give a good idea of the problem

1. Date formatting error (here due to wrong culture)

Problem The ReaderException is meaningless in that case, the IReader/IParser infos are very confusing for novices and actually need to go grab the meaningful message from the InnerException :
---> System.FormatException: String '21/2/2024' was not recognized as a valid DateOnly.

Current Error message:

CsvHelper.ReaderException: An unexpected error occurred.
IReader state:
   ColumnCount: 0
   CurrentIndex: 12
   HeaderRecord:
["Types","Chinese names","Names","Codes","Gross px (CNH)","Net Px (CNH)","Total shares","Notional (CNH)","IA (Nomura pay)","Fixed Rate (p.a.)","Trade Date","Effective Date","Valuation Date","Scheduled Termination Date","Cash Settlement Payment Date - Valuation Date","Gross unwind px (CNH)","Final px (CNH)","Settlement Currency","FX","Qty remaining","Remark","Gross px (USD)","Net Px (USD)"]
IParser state:
   ByteCount: 0
   CharCount: 519
   Row: 2
   RawRow: 2
   Count: 23
   RawRecord:
Hidden because ExceptionMessagesContainRawData is false.

Expectation (From End-user point of view) :
Get Formatting error directly instead of unexpected error occurred along with column and row ids
Example : Failed to parse value '21/2/2024' for header:"Effective Date" at row-index:0 due to FormatException

2. Missing Headers

Problem Even if the HeaderValidationException contains well identify problem at the beginning, there is an obvious verbosity issue here. The message is huge, making it scary for novice user. The Headers: are not meaningful and duplicated so many times; again the IReader/IParser infos are very confusing for novices . I ended up removing everything from 1st "Headers:" for display which is not ideal of course.

Expectation (From End-user point of view) : Get a minimalist error message
Example : Some Headers were not found ['Code', 'Shares' , 'Total', ...]

Current Error message:

CsvHelper.HeaderValidationException: Header with name 'Code'[0] was not found.
Header with name 'Shares'[0] was not found.
Header with name 'Total'[0] was not found.
Header with name 'B/S'[0] was not found.
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
Headers: 'Types', 'Chinese names', 'Names', 'Codes', 'Gross px (CNH)', 'Net Px (CNH)', 'Total shares', 'Notional (CNH)', 'IA (Nomura pay)', 'Fixed Rate (p.a.)', 'Trade Date', 'Effective Date', 'Valuation Date', 'Scheduled Termination Date', 'Cash Settlement Payment Date - Valuation Date', 'Gross unwind px (CNH)', 'Final px (CNH)', 'Settlement Currency', 'FX', 'Qty remaining', 'Remark', 'Gross px (USD)', 'Net Px (USD)'
If you are expecting some headers to be missing and want to ignore this validation, set the configuration HeaderValidated to null. You can also change the functionality to do something else, like logging the issue.

IReader state:
   ColumnCount: 0
   CurrentIndex: -1
   HeaderRecord:
["Types","Chinese names","Names","Codes","Gross px (CNH)","Net Px (CNH)","Total shares","Notional (CNH)","IA (Nomura pay)","Fixed Rate (p.a.)","Trade Date","Effective Date","Valuation Date","Scheduled Termination Date","Cash Settlement Payment Date - Valuation Date","Gross unwind px (CNH)","Final px (CNH)","Settlement Currency","FX","Qty remaining","Remark","Gross px (USD)","Net Px (USD)"]
IParser state:
   ByteCount: 0
   CharCount: 346
   Row: 1
   RawRow: 1
   Count: 23
   RawRecord:
Hidden because ExceptionMessagesContainRawData is false.