Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add node event dispatcher #1523

Merged
merged 2 commits into from
Sep 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions templates/lib/yarp/node.rb.erb
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,31 @@ module YARP
}.compact.join(", ") %>]
end

# def compact_child_nodes: () -> Array[Node]
def compact_child_nodes
<%- if node.fields.any? { |field| field.is_a?(YARP::OptionalNodeField) } -%>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another possibility would be to use [node_field, *optional_node_field, *node_list_field] because there is no #to_a on Node and *nil is [].
Not sure if faster or slower though, it would make the logic here a bit simpler.

compact = []
<%- node.fields.each do |field| -%>
<%- case field -%>
<%- when YARP::NodeField -%>
compact << <%= field.name %>
<%- when YARP::OptionalNodeField -%>
compact << <%= field.name %> if <%= field.name %>
<%- when YARP::NodeListField -%>
compact.concat(<%= field.name %>)
<%- end -%>
<%- end -%>
compact
<%- else -%>
[<%= node.fields.map { |field|
case field
when YARP::NodeField then field.name
when YARP::NodeListField then "*#{field.name}"
end
}.compact.join(", ") %>]
<%- end -%>
end

# def comment_targets: () -> Array[Node | Location]
def comment_targets
[<%= node.fields.map { |field|
Expand Down Expand Up @@ -136,6 +161,13 @@ module YARP
<%- end -%>
inspector.to_str
end

# Returns a symbol representation of the type of node.
#
# def human: () -> Symbol
def human
:<%= node.human %>
end
end

<%- end -%>
Expand All @@ -157,6 +189,55 @@ module YARP
<%- end -%>
end

# The dispatcher class fires events for nodes that are found while walking an AST to all registered listeners. It's
# useful for performing different types of analysis on the AST without having to repeat the same visits multiple times
class Dispatcher
# attr_reader listeners: Hash[Symbol, Array[Listener]]
attr_reader :listeners

def initialize
@listeners = {}
end

# Register a listener for one or more events
#
# def register: (Listener, *Symbol) -> void
def register(listener, *events)
events.each { |event| (listeners[event] ||= []) << listener }
end

# Walks `root` dispatching events to all registered listeners
#
# def dispatch: (Node) -> void
def dispatch(root)
queue = [root]

while (node = queue.shift)
case node.human
<%- nodes.each do |node| -%>
when :<%= node.human %>
listeners[:<%= node.human %>_enter]&.each { |listener| listener.<%= node.human %>_enter(node) }
queue = node.compact_child_nodes.concat(queue)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the correct order?
Maybe it should be queue.concat(node.compact_child_nodes)?

It looks like this adds child nodes before the current elements in the array (so on the "left side" i.e. near 0 index) and queue.shift above removes from the "left side" too.
So it looks like it behaves somewhat like a stack:

q = [1]
q.shift

q = [2, 3].concat(q)
p q # => [2, 3]
p q.shift # => 2

q = [4].concat(q)
p q # => [4, 3]
p q.shift # => 4, the last added element is the first out in this case
# Not fully LIFO though because if it was `q = [4, 5].concat(q)` above, then 4 is out before 5

listeners[:<%= node.human %>_leave]&.each { |listener| listener.<%= node.human %>_leave(node) }
<%- end -%>
end
Comment on lines +216 to +223
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These big case/when/end statements (dispatch is 570 lines) are pretty bad for optimizing JITs like on TruffleRuby and JRuby, it's likely too much code in a single method to compile or optimize well, somewhat similar to whitequark/parser#871.
Also this relies very heavily on case expr; when :symbol; when :symbol2; ...; end to use a Hash internally, otherwise it would be very slow as it would test each Symbol in order. I think that's the case on CRuby but on no other Ruby implementation.

I think something that would be fast for all Rubies and behave better for JIT would be to use a Hash of node.human to Procs containing the body of each when (one per node type to avoid needing listerner.send which would be slow).

end
end

# Dispatches a single event for `node` to all registered listeners
#
# def dispatch_once: (Node) -> void
def dispatch_once(node)
case node.human
<%- nodes.each do |node| -%>
when :<%= node.human %>
listeners[:<%= node.human %>_enter]&.each { |listener| listener.<%= node.human %>_enter(node) }
listeners[:<%= node.human %>_leave]&.each { |listener| listener.<%= node.human %>_leave(node) }
<%- end -%>
end
end
end

module DSL
private

Expand Down
39 changes: 39 additions & 0 deletions test/yarp/dispatcher_test.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# frozen_string_literal: true

require_relative "test_helper"

module YARP
class DispatcherTest < TestCase
class TestListener
attr_reader :events_received

def initialize
@events_received = []
end

def call_node_enter(node)
events_received << :call_node_enter
end

def call_node_leave(node)
events_received << :call_node_leave
end
end

def test_dispatching_events
listener = TestListener.new

dispatcher = Dispatcher.new
dispatcher.register(listener, :call_node_enter, :call_node_leave)

root = YARP.parse(<<~RUBY).value
def foo
something(1, 2, 3)
end
RUBY

dispatcher.dispatch(root)
assert_equal([:call_node_enter, :call_node_leave], listener.events_received)
end
end
end