Skip to content

Handle missing costs and quantities #1490

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 3, 2025
Merged

Conversation

flanakin
Copy link
Collaborator

@flanakin flanakin commented Apr 9, 2025

🛠️ Description

Address new data quality issues with data ingested into Data Explorer:

  • Fix x_EffectiveUnitPrice when it's calculated and there is a rounding error compared to x_BilledUnitPrice or ContractedUnitPrice.
  • Calculate PricingQuantity and ConsumedQuantity when there is cost but no quantity.
  • Set ListCost based on ContractedCost or ListUnitPrice when not specified.
  • Replaced "-2" and "Unassigned" values in the x_InvoiceSectionId and x_InvoiceSectionName columns.
  • Corrected x_EffectiveUnitPrice when it's calculated and has a rounding error.
  • Add new x_SourceChanges checks for "MissingConsumedQuantity", "MissingPricingQuantity", and "XEffectiveUnitPriceRoundingError".

Related to #1111

📋 Checklist

🔬 How did you test this change?

  • 🤏 Lint tests
  • 🤞 PS -WhatIf / az validate
  • 👍 Manually deployed + verified
  • 💪 Unit tests
  • 🙌 Integration tests

🙋‍♀️ Do any of the following that apply?

  • 🚨 This is a breaking change.
  • 🤏 The change is less than 20 lines of code.

📑 Did you update docs/changelog.md?

  • ✅ Updated changelog (required for dev PRs)
  • ➡️ Will add log in a future PR (feature branch PRs only)
  • ❎ Log not needed (small/internal change)

📖 Did you update documentation?

  • ✅ Public docs in docs (required for dev)
  • ✅ Internal dev docs in src (required for dev)
  • ➡️ Will add docs in a future PR (feature branch PRs only)
  • ❎ Docs not needed (small/internal change)

@flanakin flanakin added this to the 2025-04 - April milestone Apr 9, 2025
@microsoft-github-policy-service microsoft-github-policy-service bot added the Needs: Review 👀 PR that is ready to be reviewed label Apr 9, 2025
@flanakin flanakin enabled auto-merge (squash) April 9, 2025 05:50
@flanakin flanakin added the Tool: FinOps hubs Data pipeline solution label Apr 9, 2025
@flanakin flanakin disabled auto-merge May 3, 2025 17:16
@flanakin flanakin merged commit 4042d80 into dev May 3, 2025
2 checks passed
@flanakin flanakin deleted the flanakin/dev/hubs-dq-missingcq branch May 3, 2025 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs: Review 👀 PR that is ready to be reviewed Tool: FinOps hubs Data pipeline solution
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants