Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cp opcodes #2192

Closed
wants to merge 7 commits into from
Closed

Cp opcodes #2192

wants to merge 7 commits into from

Conversation

aarnaty
Copy link
Contributor

@aarnaty aarnaty commented Jan 27, 2025

#2864 Made additional changes to Opcodes.enum and related files as discussed.

…gOperation. Added row and col test cases for the former.
# Conflicts:
#	src/main/java/org/apache/sysds/hops/rewrite/RewriteAlgebraicSimplificationDynamic.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewritePushdownSumOnBinaryTest.java
# Conflicts:
#	src/main/java/org/apache/sysds/common/Opcodes.java
#	src/main/java/org/apache/sysds/common/Types.java
#	src/main/java/org/apache/sysds/hops/FunctionOp.java
#	src/main/java/org/apache/sysds/hops/codegen/cplan/CNodeBinary.java
#	src/main/java/org/apache/sysds/hops/codegen/cplan/CNodeUnary.java
#	src/main/java/org/apache/sysds/hops/cost/CostEstimator.java
#	src/main/java/org/apache/sysds/hops/cost/CostEstimatorStaticRuntime.java
#	src/main/java/org/apache/sysds/hops/rewrite/HopRewriteUtils.java
#	src/main/java/org/apache/sysds/lops/CentralMoment.java
#	src/main/java/org/apache/sysds/lops/CoVariance.java
#	src/main/java/org/apache/sysds/lops/Compression.java
#	src/main/java/org/apache/sysds/lops/Ctable.java
#	src/main/java/org/apache/sysds/lops/Data.java
#	src/main/java/org/apache/sysds/lops/DataGen.java
#	src/main/java/org/apache/sysds/lops/DnnTransform.java
#	src/main/java/org/apache/sysds/lops/FunctionCallCP.java
#	src/main/java/org/apache/sysds/lops/GroupedAggregate.java
#	src/main/java/org/apache/sysds/lops/MMTSJ.java
#	src/main/java/org/apache/sysds/lops/MapMultChain.java
#	src/main/java/org/apache/sysds/lops/MatMultCP.java
#	src/main/java/org/apache/sysds/lops/Nary.java
#	src/main/java/org/apache/sysds/lops/OperatorOrderingUtils.java
#	src/main/java/org/apache/sysds/lops/PMMJ.java
#	src/main/java/org/apache/sysds/lops/ParameterizedBuiltin.java
#	src/main/java/org/apache/sysds/lops/PartialAggregate.java
#	src/main/java/org/apache/sysds/lops/PickByCount.java
#	src/main/java/org/apache/sysds/lops/SortKeys.java
#	src/main/java/org/apache/sysds/lops/SpoofFused.java
#	src/main/java/org/apache/sysds/lops/Sql.java
#	src/main/java/org/apache/sysds/lops/TernaryAggregate.java
#	src/main/java/org/apache/sysds/lops/Transform.java
#	src/main/java/org/apache/sysds/lops/UAggOuterChain.java
#	src/main/java/org/apache/sysds/lops/WeightedCrossEntropy.java
#	src/main/java/org/apache/sysds/lops/WeightedDivMM.java
#	src/main/java/org/apache/sysds/lops/WeightedSigmoid.java
#	src/main/java/org/apache/sysds/lops/WeightedSquaredLoss.java
#	src/main/java/org/apache/sysds/lops/WeightedUnaryMM.java
#	src/main/java/org/apache/sysds/lops/compile/linearization/LinearizerMaxParallelism.java
#	src/main/java/org/apache/sysds/parser/Expression.java
#	src/main/java/org/apache/sysds/parser/PrintStatement.java
#	src/main/java/org/apache/sysds/parser/dml/DmlSyntacticValidator.java
#	src/main/java/org/apache/sysds/resource/cost/CPCostUtils.java
#	src/main/java/org/apache/sysds/resource/cost/CostEstimator.java
#	src/main/java/org/apache/sysds/resource/cost/SparkCostUtils.java
#	src/main/java/org/apache/sysds/runtime/compress/CompressedMatrixBlock.java
#	src/main/java/org/apache/sysds/runtime/frame/data/columns/OptionalArray.java
#	src/main/java/org/apache/sysds/runtime/frame/data/columns/RaggedArray.java
#	src/main/java/org/apache/sysds/runtime/instructions/CPInstructionParser.java
#	src/main/java/org/apache/sysds/runtime/instructions/InstructionUtils.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/AggregateBinaryCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/AggregateTernaryCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/AggregateUnaryCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/AppendCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/BinaryFrameFrameCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/BinaryScalarScalarCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/BuiltinNaryCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/CentralMomentCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/CovarianceCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/CtableCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/DataGenCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/DnnCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/FrameIndexingCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/IndexingCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/ListAppendRemoveCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/ListIndexingCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/MMTSJCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/MatrixBuiltinNaryCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/MatrixIndexingCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/MultiReturnBuiltinCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/MultiReturnComplexMatrixBuiltinCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/MultiReturnParameterizedBuiltinCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/PMMJCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/ParameterizedBuiltinCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/QuantilePickCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/QuaternaryCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/ReorgCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/ReshapeCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/ScalarBuiltinNaryCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/StringInitCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/UnaryFrameCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/UnaryScalarCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cp/VariableCPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/cpfile/MatrixIndexingCPFileInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/spark/RandSPInstruction.java
#	src/main/java/org/apache/sysds/runtime/instructions/spark/functions/ReorgMapFunction.java
#	src/main/java/org/apache/sysds/runtime/instructions/spark/utils/RDDAggregateUtils.java
#	src/main/java/org/apache/sysds/runtime/lineage/Lineage.java
#	src/main/java/org/apache/sysds/runtime/lineage/LineageCache.java
#	src/main/java/org/apache/sysds/runtime/lineage/LineageCacheConfig.java
#	src/main/java/org/apache/sysds/runtime/lineage/LineageItemUtils.java
#	src/main/java/org/apache/sysds/runtime/lineage/LineageRecomputeUtils.java
#	src/main/java/org/apache/sysds/runtime/lineage/LineageRewriteReuse.java
#	src/main/java/org/apache/sysds/runtime/matrix/data/LibCommonsMath.java
#	src/main/java/org/apache/sysds/runtime/matrix/data/LibMatrixCuDNN.java
#	src/main/java/org/apache/sysds/runtime/matrix/data/LibMatrixCuDNNRnnAlgorithm.java
#	src/main/java/org/apache/sysds/runtime/matrix/data/LibMatrixDNNLSTM.java
#	src/main/java/org/apache/sysds/runtime/matrix/data/MatrixBlock.java
#	src/main/java/org/apache/sysds/runtime/matrix/operators/QuaternaryOperator.java
#	src/test/java/org/apache/sysds/test/applications/ScalableDecompositionTest.java
#	src/test/java/org/apache/sysds/test/component/compress/AbstractCompressedUnaryTests.java
#	src/test/java/org/apache/sysds/test/component/compress/colgroup/ColGroupTest.java
#	src/test/java/org/apache/sysds/test/component/compress/lib/CLALibCompAggTest.java
#	src/test/java/org/apache/sysds/test/component/matrix/MatrixAggKernelTest.java
#	src/test/java/org/apache/sysds/test/component/resource/CPCostUtilsTest.java
#	src/test/java/org/apache/sysds/test/component/resource/RecompilationTest.java
#	src/test/java/org/apache/sysds/test/functions/aggregate/AggregateNaNTest.java
#	src/test/java/org/apache/sysds/test/functions/aggregate/ColSumsSqTest.java
#	src/test/java/org/apache/sysds/test/functions/aggregate/ColVariancesTest.java
#	src/test/java/org/apache/sysds/test/functions/aggregate/RowSumsSqTest.java
#	src/test/java/org/apache/sysds/test/functions/aggregate/RowVariancesTest.java
#	src/test/java/org/apache/sysds/test/functions/aggregate/SumSqTest.java
#	src/test/java/org/apache/sysds/test/functions/append/RBindCBindMatrixTest.java
#	src/test/java/org/apache/sysds/test/functions/async/AsyncBroadcastTest.java
#	src/test/java/org/apache/sysds/test/functions/async/PrefetchRDDTest.java
#	src/test/java/org/apache/sysds/test/functions/async/ReuseAsyncOpTest.java
#	src/test/java/org/apache/sysds/test/functions/binary/matrix/UDFBackwardsCompatibilityTest.java
#	src/test/java/org/apache/sysds/test/functions/binary/matrix_full_cellwise/FullSortedOuterCompareTest.java
#	src/test/java/org/apache/sysds/test/functions/binary/matrix_full_cellwise/FullVectorVectorCellwiseCompareOperationTest.java
#	src/test/java/org/apache/sysds/test/functions/binary/matrix_full_cellwise/FullVectorVectorCellwiseOperationTest.java
#	src/test/java/org/apache/sysds/test/functions/binary/matrix_full_other/FullMatrixMultiplicationTransposeSelf2Test.java
#	src/test/java/org/apache/sysds/test/functions/builtin/part1/BuiltinGridSearchTest.java
#	src/test/java/org/apache/sysds/test/functions/builtin/part2/BuiltinRaSelectionTest.java
#	src/test/java/org/apache/sysds/test/functions/codegen/CellwiseTmplTest.java
#	src/test/java/org/apache/sysds/test/functions/codegen/MiscPatternTest.java
#	src/test/java/org/apache/sysds/test/functions/codegen/OuterProdTmplTest.java
#	src/test/java/org/apache/sysds/test/functions/codegen/RowAggTmplTest.java
#	src/test/java/org/apache/sysds/test/functions/codegenalg/partone/AlgorithmAutoEncoder.java
#	src/test/java/org/apache/sysds/test/functions/codegenalg/partone/AlgorithmL2SVM.java
#	src/test/java/org/apache/sysds/test/functions/io/ScalarIOTest.java
#	src/test/java/org/apache/sysds/test/functions/lineage/CacheEvictionTest.java
#	src/test/java/org/apache/sysds/test/functions/lineage/GPULineageCacheEvictionTest.java
#	src/test/java/org/apache/sysds/test/functions/lineage/LineageReuseEvalTest.java
#	src/test/java/org/apache/sysds/test/functions/misc/FunctionPotpourriTest.java
#	src/test/java/org/apache/sysds/test/functions/misc/IPAConstantFoldingScalarVariablePropagationTest.java
#	src/test/java/org/apache/sysds/test/functions/misc/IPADeadCodeEliminationTest.java
#	src/test/java/org/apache/sysds/test/functions/misc/IPAFunctionInliningTest.java
#	src/test/java/org/apache/sysds/test/functions/recompile/LiteralReplaceCastScalarReadTest.java
#	src/test/java/org/apache/sysds/test/functions/recompile/RemoveEmptyRecompileTest.java
#	src/test/java/org/apache/sysds/test/functions/reorg/FullReverseTest.java
#	src/test/java/org/apache/sysds/test/functions/reorg/FullRollTest.java
#	src/test/java/org/apache/sysds/test/functions/reorg/MultipleOrderByColsTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteBinaryMV2OuterTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteCSETransposeScalarTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteCTableToRExpandTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteConstantConjunctionDisjunctionTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteCumsumConstantTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteDistributiveMatrixMultTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteElementwiseMultChainOptimizationAllTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteEliminateRemoveEmptyTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteFoldMinMaxTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteFoldRCBindTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteForLoopRemovalTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteFuseBinaryOpChainTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteFuseBinarySubDAGToUnaryOperationTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteHoistingLoopInvariantOpsTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteListTsmmCVTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteMatrixMultChainOptSparseTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteMatrixMultChainOptTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteMergeBlocksTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteNaryMultTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteNotOverComparisonsTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewritePullupAbsTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewritePushdownUaggTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteRemoveComparisonChainsTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteRemoveUnnecessaryRevTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteSimplifyBushyBinaryOperationTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteSimplifyConstantSortTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteSimplifyEmptyColMeansTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteSimplifyLowerTriExtractionTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteSimplifyNnzComputationTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteSimplifyOrderedSortTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteSimplifyRowColSumMVMultTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteSimplifySumDiagToTraceTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteSimplifyTraceMatrixMultTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteSimplifyWeightedSigmoidMMChainsTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteSimplifyWeightedSquaredLossTest.java
#	src/test/java/org/apache/sysds/test/functions/rewrite/RewriteSimplifyWeightedUnaryMMTest.java
#	src/test/java/org/apache/sysds/test/functions/ternary/ABATernaryAggregateTest.java
#	src/test/java/org/apache/sysds/test/functions/ternary/CTableSequenceTest.java
#	src/test/java/org/apache/sysds/test/functions/ternary/TernaryAggregateTest.java
#	src/test/java/org/apache/sysds/test/functions/unary/matrix/FullSignTest.java
@mboehm7
Copy link
Contributor

mboehm7 commented Feb 1, 2025

Thanks for the patch @aarnaty - during the merge, I fixed the conflicts (1000s of them), unnecessary imports, and formatting issues. In the future, please start from a clean branch and rebase to main.

@mboehm7 mboehm7 closed this in 98c2446 Feb 1, 2025
saminbassiri pushed a commit to saminbassiri/systemds that referenced this pull request Feb 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants