ERROR: Could not find a valid gem 'logstash-devutils' (= 0.0.15) in any repository

[michaelhidalgo]

Hi guys, I'm newbie to ruby as you will noticed but I wanted to install the logstash-devutils gem (https://rubygems.org/gems/logstash-devutils/versions/0.0.15-java) on the Mac following this command :

gem install --local logstash-devutils -v 0.0.15

And i get below error message:
:
ERROR: Could not find a valid gem 'logstash-devutils' (= 0.0.15) in any repository

Any thoughts?

I also tried latest version of the gem
gem install logstash-devutils
ERROR: Could not find a valid gem 'logstash-devutils' (>= 0), here is why:
Found logstash-devutils (1.3.1), but was for platform java

[michaelhidalgo]

Update: I downloaded the gem manually and when I do try to install it manually, i get a dependency errors

Unable to resolve dependencies: logstash-devutils requires logstash-core-plugin-api (<= 2.99, >= 2.0)

[jordansissel]

This is failing to install because the ruby you are using is not JRuby.

I have a broader question, though, what makes you try to install logstash-devutils manually? Our intention is that this library provides support for logstash tests and logstash plugin tests. You generally do not need to install it manually.

What are you trying to do that makes you want to install logstash-devutils manually?

[michaelhidalgo]

Thanks for the feedback, I do have JRuby but the issue seems to be a dependency hell.

In regards to your question: I'm trying to find a way to create a logstash configuration (including customized grok patterns) and be able to provide a unit test of the configuration file. This is to ensure quality when providing a custom configuration file.

In your case, you have some grok patterns that does not seem to have a test associated to it, so I want to implement a mechanism to provide a test scenario that can be used to find issues on the patterns and so on.

To give you a concrete example, please check this out this cool hack https://github.com/gquintana/gquintana.github.io/blob/master/sources/2016-09-07-Testing-Logstash-configuration/spec/02_logstash_nginx_filter_spec.rb#L2

As you can see on line number 2, the guy is using require "logstash/devutils/rspec/spec_helper"
and I wanted to replace that import with the gem (i.e be able to reuse your spec helper without needing the full project)

Any ideas how to address that issue?

[jordansissel]

I do have JRuby but the issue seems to be a dependency hell.

I don't know what you mean by dependency hell.

I also tried latest version of the gem
gem install logstash-devutils
ERROR: Could not find a valid gem 'logstash-devutils' (>= 0), here is why:
Found logstash-devutils (1.3.1), but was for platform java

This error message is an indication that you are running gem under MRI (regular ruby, not jruby). The clue for me is that it says "Found logstash-devutils (1.3.1), but was for platform java" which means your gem is not running under jruby.

In general, you do not need to install logstash-devutils manually, ever. I personally haven't installed this gem manually in a very long time. Our dependency listing includes it (in plugins and in logstash itself) and you will get it automatically.

I wanted to replace that import with the gem

(Pardon the detailed explanation below if you already know these things, I'm trying to get you a good answer)

In ruby, require is a function that loads a ruby file. A gem provides these ruby files and instructs the ruby runtime about paths where ruby files live.

I looked at what your link, and I think I understand a little bit more now.

What you linked has a test.sh that uses a file we no longer ship with main logstash releases (bin/rspec). In order to run this, you may be best getting logstash setup from source:

• Clone the logstash repo, and run the Building Logstash steps
• Then run the test.sh from that source you linked.

I do not believe there are shortcuts to this. If you want a shortcut, can you help me understand what you want to avoid?

[michaelhidalgo]

Hi @jordansissel , thanks for the further explanation, it is very useful specially since I'm a newcomer to ruby itself.

What I meant by dependency hell, is if you look at the gem published here https://rubygems.org/gems/logstash-devutils/versions/1.3.1-java, it says it has many dependencies and even when I tried to install the gem including all the dependencies (gem install --include-dependencies) I got some warnings that dependencies were not found. Anyway I'm pretty sure I was doing something wrong when installing the gem and need to understand better the switch from ruby to jruby.

I was expecting to run the unit testing isolated without needing logstash repo, that is for convenience. Basically the use case is that we can write logstash configurations for several purposes and for parsing log files but provide also a unit test with a sample log, something similar to what you saw in the example.

Since the gem was published, I was expecting that you could use it in your project without relying totally on the logstash project. I certainly can test just the grok pattern, but I'm more concern on testing the output of the logstash file after doing several mutations and applying filters.

As you can see basically the use case is to provide a Logstash configuration file for any specific purpose but also provide a unit test with a sample log that can be used for troubleshooting purposes as versions of the log file can be different.

So in order to answer your question, I'm looking for a way to distribute our configurations and unit testing without needing logstash code although it might not be possible as I can see.

I will follow your recommendations about cloning the Logstash repo and building it in the way you have explained.

Thanks for all your help.

[jslirola]

@michaelhidalgo Did you fix the issue? I'm trying to follow the basic tutorial to write a logstash filter plugin but I am not able to install my example:

$ bundler install
Fetching gem metadata from https://rubygems.org/.........
Could not find gem 'logstash-devutils' in any of the gem sources listed in your Gemfile.

Thank you.

[pedropereiractw]

Hi anyone found a solution for this problem? I'm using JRuby and even after I install manualy the 'logstash-devutils' I can't run the bundle install:

Fetching gem metadata from https://rubygems.org/......... Could not find gem 'logstash-devutils' in any of the gem sources listed in your Gemfile.

[jsvd]

I'm having trouble replicating this, what jruby are you using? can you post ruby -v ?

[pedropereiractw]

$ jruby -v jruby 9.2.5.0 (2.5.0) 2018-12-06 6d5a228 Java HotSpot(TM) 64-Bit Server VM 25.191-b12 on 1.8.0_191-b12 +jit [mswin32-x86_64]

[jsvd]

can you do which bundler? are you sure the bundler you're running is the one installed in jruby?

[pedropereiractw]

I'm running it like jruby -S bundle install and if I runn jruby -S gem list I can find this entry: logstash-devutils (1.3.1 java)

[jsvd]

is bundle installed locally in that jruby installation? -S will make jruby look for the "script" (bundle/gem) in bin or the PATH. so it's not guaranteed that using -S will make it use the jruby's bundle.

I'm having trouble finding any other reason other than jruby/bundler/gem weirdness since I'm able to run that command locally and it's a normal gem installation like any other

[pedropereiractw]

So I uninstall every ruby version of my machine, reinstall only JRuby and I'm using only bundle install and kept bump into the same error, here is my gem env:

`RubyGems Environment:

• RUBYGEMS VERSION: 2.7.6
• RUBY VERSION: 2.5.0 (2018-12-06 patchlevel 0) [java]
• INSTALLATION DIRECTORY: C:/jruby-9.2.5.0/lib/ruby/gems/shared
• USER INSTALLATION DIRECTORY: C:/Users/ctw00220/.gem/jruby/2.5.0
• RUBY EXECUTABLE: C:/jruby-9.2.5.0/bin/jruby.exe
• EXECUTABLE DIRECTORY: C:/jruby-9.2.5.0/bin
• SPEC CACHE DIRECTORY: C:/Users/ctw00220/.gem/specs
• SYSTEM CONFIGURATION DIRECTORY: C:/jruby-9.2.5.0/etc
• RUBYGEMS PLATFORMS:
  • ruby
  • universal-java-1.8
• GEM PATHS:
  • C:/jruby-9.2.5.0/lib/ruby/gems/shared
  • C:/Users/ctw00220/.gem/jruby/2.5.0
• GEM CONFIGURATION:
  • :update_sources => true
  • :verbose => true
  • :backtrace => false
  • :bulk_threshold => 1000
  • "install" => "--no-rdoc --no-ri --env-shebang"
  • "update" => "--no-rdoc --no-ri --env-shebang"
• REMOTE SOURCES:
  • https://rubygems.org/
• SHELL PATH:
  • C:\Program Files\ConEmu\ConEmu\Scripts
  • C:\Program Files\ConEmu
  • C:\Program Files\ConEmu\ConEmu
  • C:\ProgramData\DockerDesktop\version-bin
  • C:\Program Files\Docker\Docker\Resources\bin
  • C:\Program Files (x86)\Common Files\Oracle\Java\javapath
  • C:\Windows\system32
  • C:\Windows
  • C:\Windows\System32\Wbem
  • C:\Windows\System32\WindowsPowerShell\v1.0\
  • C:\Windows\System32\OpenSSH\
  • C:\Program Files\Intel\WiFi\bin\
  • C:\Program Files\Common Files\Intel\WirelessCommon\
  • C:\apache-maven-3.6.0\bin
  • C:\Program Files\Java\jdk1.8.0_191\bin
  • C:\Program Files\nodejs\
  • C:\ProgramData\chocolatey\bin
  • C:\HashiCorp\Vagrant\bin
  • C:\Program Files (x86)\Yarn\bin\
  • c:\jruby-9.2.5.0\bin`

[pedropereiractw]

I switch to an Ubuntu machine and install JRuby with RVM, when I run:

ruby -v

I get:

jruby 9.2.0.0 (2.5.0) 2018-05-24 81156a8 Java HotSpot(TM) 64-Bit Server VM 25.191-b12 on 1.8.0_191-b12 +jit [linux-x86_64]

When I try to run bundle install I kept getting the same error:

Fetching gem metadata from https://rubygems.org/........ Could not find gem 'logstash-devutils unknown' in any of the gem sources listed in your Gemfile.

My Gem ENV:

`RubyGems Environment:

• RUBYGEMS VERSION: 2.7.6
• RUBY VERSION: 2.5.0 (2018-05-24 patchlevel 0) [java]
• INSTALLATION DIRECTORY: /home/vagrant/.rvm/gems/jruby-9.2.0.0
• USER INSTALLATION DIRECTORY: /home/vagrant/.gem/jruby/2.5.0
• RUBY EXECUTABLE: /usr/share/rvm/rubies/jruby-9.2.0.0/bin/jruby
• EXECUTABLE DIRECTORY: /home/vagrant/.rvm/gems/jruby-9.2.0.0/bin
• SPEC CACHE DIRECTORY: /home/vagrant/.gem/specs
• SYSTEM CONFIGURATION DIRECTORY: /usr/share/rvm/rubies/jruby-9.2.0.0/etc
• RUBYGEMS PLATFORMS:
  • ruby
  • universal-java-1.8
• GEM PATHS:
  • /home/vagrant/.rvm/gems/jruby-9.2.0.0
  • /home/vagrant/.rvm/gems/jruby-9.2.0.0@global
• GEM CONFIGURATION:
  • :update_sources => true
  • :verbose => true
  • :backtrace => false
  • :bulk_threshold => 1000
  • "install" => "--no-rdoc --no-ri"
  • "update" => "--no-rdoc --no-ri"
• REMOTE SOURCES:
  • https://rubygems.org/
• SHELL PATH:
  • /home/vagrant/.rvm/gems/jruby-9.2.0.0/bin
  • /home/vagrant/.rvm/gems/jruby-9.2.0.0@global/bin
  • /usr/share/rvm/rubies/jruby-9.2.0.0/bin
  • /usr/share/rvm/bin
  • /home/vagrant/bin
  • /home/vagrant/.local/bin
  • /usr/local/sbin
  • /usr/local/bin
  • /usr/sbin
  • /usr/bin
  • /sbin
  • /bin
  • /usr/games
  • /usr/local/games
  • /snap/bin
  • /usr/lib/jvm/java-8-oracle/bin
  • /usr/lib/jvm/java-8-oracle/db/bin
  • /usr/lib/jvm/java-8-oracle/jre/bin`

Any lights about this?

[pedropereiractw]

Solved it adding to my gemspecs the follow:

s.platform = Gem::Platform::JAVA if defined?(RUBY_ENGINE) && RUBY_ENGINE == 'jruby'