Integrate PSLab-Python with OSS-Fuzz for Continuous Fuzz Testing

[Shivam7-1]

Hello @bessman ,

I hope this message finds you well! I’d like to propose integrating PSLab-Python with Google's OSS-Fuzz continuous fuzz testing platform designed to improve the stability and security of open-source projects.

Why Integrate PSLab-Python with OSS-Fuzz?

1. Improved Stability: OSS-Fuzz can automatically detect edge-case bugs, crashes, and security vulnerabilities in PSLab-Python’s codebase.
2. Enhanced Reliability: Continuous fuzzing ensures that untrusted inputs, such as data from sensors, communication protocols, or user-defined configurations, are handled robustly.
3. Proactive Bug Fixes: By identifying potential issues early, OSS-Fuzz helps maintain a stable and secure codebase.

Benefits for PSLab-Python

• Continuous Testing: OSS-Fuzz runs automated fuzz tests continuously, providing ongoing feedback about the project’s stability.
• Bug Reporting: OSS-Fuzz identifies and reports any issues directly to the maintainers, allowing for timely fixes.
• Improved User Confidence: Ensuring robustness against unexpected inputs enhances trust in PSLab-Python’s reliability.

I’d be happy to work for setting up the integration. Let me know Can I move Forward with this And Create PR for Same
Looking forward to your feedback!
Best regards

[bessman]

Hi @Shivam7-1. I'd actually never heard of fuzz testing before, but it seems like it could be useful. Go ahead and work on integrating OSS-Fuzz with pslab-python!

I'm curious how you plan to approach the issue of serial traffic emulation. Most of what pslab-python does involves talking to a pslab device over a serial bus, which makes testing a bit tricky. The current testing approach relies on replaying recorded serial traffic, but this has resulted in a very high maintenance overhead and I would like to move away from this approach.

Does OSS-Fuzz offer an alternative way to test?

[Shivam7-1]

Hii @bessman Thanks For Responding
To address the serial traffic emulation challenge, I plan to create fuzz targets that mock PSLab hardware interactions like set_gain, analog_read, and send_data. This approach will emulate hardware behavior without relying on actual devices or recorded traffic, leveraging OSS-Fuzz to test software logic effectively.

Let me know if you have any suggestions!
Best regards,
Shivam

[bessman]

Sounds good!

As a first step, I would suggest implementing a fuzz test for a single function, as a proof of concept. For example, triggering an oscilloscope read with pslab.oscilloscope.capture. This would allow me to better understand the concept and evaluate its suitability for pslab-python.

[Shivam7-1]

Hii @bessman Thanks For Reply
here is below Code which could work

import sys
import atheris
import pslab

def TestOneInput(data):
    fdp = atheris.FuzzedDataProvider(data)

    try:
        # Initialize the communication interfaces
        i2c = pslab.i2c.I2C()

        # Fuzz I2C read/write operations
        address = fdp.ConsumeIntInRange(0, 127)  # Valid 7-bit I2C addresses
        register = fdp.ConsumeIntInRange(0, 255)  # Register range
        value = fdp.ConsumeIntInRange(0, 255)  # Data to write
        
        # Fuzz reading from an I2C device
        i2c.read(address, register)
        
        # Fuzz writing to an I2C device
        i2c.write(address, register, value)

        # Fuzz oscilloscope functionality
        oscilloscope = pslab.oscilloscope.Oscilloscope()
        channel = fdp.ConsumeIntInRange(1, 4)  # Channels 1-4
        timebase = fdp.ConsumeFloat()  # Timebase for capture
        samples = fdp.ConsumeIntInRange(1, 1000)  # Number of samples
        oscilloscope.capture(channel=channel, timebase=timebase, samples=samples)
    except Exception:
        pass

def main():
    atheris.Setup(sys.argv, TestOneInput)
    atheris.Fuzz()

if __name__ == "__main__":
    main()

How about this for pslab_fuzz.py file As in ossfuzz Continous fuzzing and auto Alert would be Given As well as its Scan whole Repo code so it would automaticallygive alerts if it finds anything missing
Thanks

[Shivam7-1]

Here is Another varient of Code which we could also use

import atheris
import sys
from pslab import ScienceLab

def TestOneInput(data):
    fdp = atheris.FuzzedDataProvider(data)
    try:
        # Initialize the ScienceLab instance
        psl = ScienceLab()

        capacitance_range = fdp.ConsumeIntInRange(1, 5)  
        psl.multimeter.measure_capacitance(capacitance_range)

        channel = fdp.ConsumeIntInRange(1, 4)  
        samples = fdp.ConsumeIntInRange(100, 1000)  # Number of samples
        timebase = fdp.ConsumeFloat()  # Timebase for the capture
        psl.oscilloscope.capture(channel=channel, samples=samples, timebase=timebase)

        frequency = fdp.ConsumeIntInRange(1, 5000)  # Frequency in Hz
        amplitude = fdp.ConsumeIntInRange(1, 5)  # Amplitude in volts
        psl.waveform_generator.generate_wave(frequency=frequency, amplitude=amplitude)
    except Exception as e:
        # Log unexpected errors for debugging
        if "expected exception" not in str(e):
            raise

def main():
    atheris.Setup(sys.argv, TestOneInput)
    atheris.Fuzz()

if __name__ == "__main__":
    main()

As same oss fuzz gives Scan whole Repo code so it would automatically give alerts if it finds anything missing
if i am missing with parameters could you give me some like psl.oscilloscope.capture
Thanks

[bessman]

That looks better. A couple of things:

1. It's pslab.Oscilloscope(), or pslab.instrument.Oscilloscope(), or pslab.instrument.oscilloscope.Oscilloscope(), not pslab.oscilloscope.Oscilloscope().
2. The parameters of Oscilloscope.capture are channels and timegap, not channel and timebase.
3. It's psl.waveform_generator.generate, not .generate_wave

I would also suggest splitting up the tests in different functions. One for I2C, one for oscilloscope, one for waveform generator, and so on.

How does one run these tests? If a test passes, what does that tell me? If it fails, what does that tell me?

[Shivam7-1]

How this below code

import sys
import atheris
import pslab


def fuzz_i2c(data):
    fdp = atheris.FuzzedDataProvider(data)
    i2c = pslab.I2C()
    try:
        address = fdp.ConsumeIntInRange(0, 127)  # Valid 7-bit I2C addresses
        register = fdp.ConsumeIntInRange(0, 255)  # Register range
        value = fdp.ConsumeIntInRange(0, 255)  # Data to write

        # Fuzz reading from an I2C device
        i2c.read(address, register)

        # Fuzz writing to an I2C device
        i2c.write(address, register, value)
    except Exception:
        pass


def fuzz_oscilloscope(data):
    fdp = atheris.FuzzedDataProvider(data)
    try:
        oscilloscope = pslab.instrument.Oscilloscope()

        # Fuzz parameters for capture
        channels = [fdp.ConsumeIntInRange(1, 4) for _ in range(2)] 
        timegap = fdp.ConsumeFloat()  # Time gap between captures

        oscilloscope.capture(channels=channels, timegap=timegap)
    except Exception:
        pass


def fuzz_waveform_generator(data):
    fdp = atheris.FuzzedDataProvider(data)
    try:
        waveform_generator = pslab.waveform_generator.WaveformGenerator()

        # Fuzz parameters for waveform generation
        frequency = fdp.ConsumeFloatInRange(1.0, 100000.0)  # Frequency in Hz
        amplitude = fdp.ConsumeFloatInRange(0.1, 5.0)  # Amplitude in volts
        waveform_type = fdp.ConsumeString(8)  # Waveform type (e.g., 'sine', 'square', 'triangle')

        waveform_generator.generate(frequency=frequency, amplitude=amplitude, waveform_type=waveform_type)
    except Exception:
        pass


def TestOneInput(data):
    fuzz_i2c(data)
    fuzz_oscilloscope(data)
    fuzz_waveform_generator(data)


def main():
    atheris.Setup(sys.argv, TestOneInput)
    atheris.Fuzz()


if __name__ == "__main__":
    main()

[bessman]

channels = [fdp.ConsumeIntInRange(1, 4) for _ in range(2)]

Why did channels become a list?

I am starting to suspect I am talking to an AI. Please carefully read the documentation and write code that will actually work, or I will close this issue.

[Shivam7-1]

hii
i think here for Two channels for multi-capture i think i had mistake
sorry this is my first time integrating project

import sys
import atheris
import pslab


def fuzz_i2c(data):
    fdp = atheris.FuzzedDataProvider(data)
    i2c = pslab.I2C()
    try:
        address = fdp.ConsumeIntInRange(0, 127)  # Valid 7-bit I2C addresses
        register = fdp.ConsumeIntInRange(0, 255)  
        value = fdp.ConsumeIntInRange(0, 255)

        i2c.read(address, register)

        i2c.write(address, register, value)
    except Exception:
        pass


def fuzz_oscilloscope(data):
    fdp = atheris.FuzzedDataProvider(data)
    try:
        oscilloscope = pslab.instrument.Oscilloscope()

        channel = fdp.ConsumeIntInRange(1, 4)
        timegap = fdp.ConsumeFloat()  # Time gap between captures

        oscilloscope.capture(channel=channel, timegap=timegap)
    except Exception:
        pass


def fuzz_waveform_generator(data):
    fdp = atheris.FuzzedDataProvider(data)
    try:
        waveform_generator = pslab.waveform_generator.WaveformGenerator()

        frequency = fdp.ConsumeFloatInRange(1.0, 100000.0)  # Frequency in Hz
        amplitude = fdp.ConsumeFloatInRange(0.1, 5.0)  # Amplitude in volts
        waveform_type = fdp.ConsumeString(8)

        waveform_generator.generate(frequency=frequency, amplitude=amplitude, waveform_type=waveform_type)
    except Exception:
        pass


def TestOneInput(data):
    fuzz_i2c(data)
    fuzz_oscilloscope(data)
    fuzz_waveform_generator(data)


def main():
    atheris.Setup(sys.argv, TestOneInput)
    atheris.Fuzz()


if __name__ == "__main__":
    main()

i think now it good to go

[bessman]

i think now it good to go

It is not. When you run this test (you do run the test yourself, right?) what is the result?

[Shivam7-1]

Everything got build sucessfully
for this here i used python3 infra/helper.py build_fuzzers

[bessman]

I'm not seeing a test result in that output. It looks like output from a build script to set up a testing environment, not from running the actual test.

[Shivam7-1]

here is sample flow ossfuzz project integration dont even need any fuzz file it do automatically scan the code of listed project time to time and accordingly if they find fault or security issue in code it will directly report to mainitainer of it
for this i had create project.yaml file you can check this here google/oss-fuzz@858dd28

[bessman]

dont even need any fuzz file it do automatically scan the code of listed project time to time and accordingly if they find fault or security issue in code it will directly report to mainitainer of it

How does it do that?

I do not understand how oss-fuzz works or what information it provides. For this issue to proceed, I need answers to the following:

1. What does a passing test look like?
2. What does it tell me?
3. What does a failing test look like?
4. What does it tell me?