Skip to content

Commit 6f981fa

Browse files
replsanjuveapen
authored andcommitted
Misc fix
1 parent 3ec197e commit 6f981fa

File tree

8 files changed

+1700040
-10
lines changed

8 files changed

+1700040
-10
lines changed

.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,4 +17,5 @@ target/*
1717
.cache
1818
logs/*
1919
.settings/
20-
checkpoint/
20+
checkpoint/
21+
.ensime_cache/

README.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,3 +25,10 @@ $ docker-compose -f kafka-docker/docker-compose.yml up -d
2525
And to shut it down:
2626
docker-compose -f kafka-docker/docker-compose.yml down
2727
The ports are 2181 for ZooKeeper and 9092 for Kafka
28+
29+
#### Run programs with sbt
30+
31+
``` shell
32+
sbt test:compile
33+
sbt runMain org.repl.kafkasparkmongo.BXBooksLoader
34+
```

build.sbt

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,21 @@
11
name := """Kakfa Spark MongoDb"""
22
version := "1.0"
33

4-
scalaVersion := "2.11.12"
4+
scalaVersion := "2.12.8"
5+
scalaVersion in ThisBuild := "2.12.8"
56

67
fork := true
78

8-
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.3.0"
9+
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.4.3"
910

10-
libraryDependencies += "org.apache.spark" %% "spark-streaming" % "2.3.0"
11+
libraryDependencies += "org.apache.spark" %% "spark-streaming" % "2.4.3"
1112

12-
libraryDependencies += "org.apache.spark" %% "spark-streaming-kafka-0-10" % "2.3.0"
13+
libraryDependencies += "org.apache.spark" %% "spark-streaming-kafka-0-10" % "2.4.3"
1314

1415
// Needed for structured streams
15-
libraryDependencies += "org.apache.spark" %% "spark-sql-kafka-0-10" % "2.3.0"
16+
libraryDependencies += "org.apache.spark" %% "spark-sql-kafka-0-10" % "2.4.3"
1617

17-
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.3.0"
18+
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.4.3"
1819

1920
libraryDependencies += "org.apache.kafka" %% "kafka" % "2.2.0"
2021

@@ -28,4 +29,4 @@ libraryDependencies += "com.fasterxml.jackson.module" %% "jackson-module-scala"
2829

2930
libraryDependencies += "com.typesafe.scala-logging" %% "scala-logging" % "3.8.0"
3031

31-
scalacOptions += "-target:jvm-1.8"
32+
scalacOptions += "-target:jvm-1.8"

0 commit comments

Comments
 (0)