Regarding PCIe peek/poke in NetBSD

[Gnanasekar Loganathan]

Hi All,

I'm new to NetBSD kernel.

In NetBSD how we can access the I/O region, Memory region and
Prefetchable region for any PCIe devices?
How to map this region and do peek/poke on those addresses?

Please advice.

Thanks,
Gnana

[Greg Troxel]

I'm new to NetBSD kernel.

In NetBSD how we can access the I/O region, Memory region and
Prefetchable region for any PCIe devices?
How to map this region and do peek/poke on those addresses?

Read the docs on www.netbsd.org, and skim the Guide. Then read/skim the
section 9 man pages, and look at example code in src/sys/dev.

[Gnanasekar Loganathan]

Hi Greg,

Thanks for the link, there i can found some useful information.

As we know that the 'pcictl' command will list all the pci devices
present in the system with corresponding device/vendor id.
The same way, is that possible to get/map the IO/Mem/prefetchable
region of any PCI device by using bus_space_map for Peek/Poke?

If so, please give guidance.

Thanks,
Gnana

 I'm new to NetBSD kernel.
 In NetBSD how we can access the I/O region, Memory region and
 Prefetchable region for any PCIe devices?
 How to map this region and do peek/poke on those addresses?
Read the docs on www.netbsd.org, and skim the Guide.  Then read/skim the
section 9 man pages, and look at example code in src/sys/dev.

[Manuel Bouyer]

Post by Gnanasekar Loganathan
Hi Greg,
Thanks for the link, there i can found some useful information.
As we know that the 'pcictl' command will list all the pci devices
present in the system with corresponding device/vendor id.
The same way, is that possible to get/map the IO/Mem/prefetchable
region of any PCI device by using bus_space_map for Peek/Poke?

I'm not sure what you want to do. Do you want to access the device
from kernel or userland ?
From kernel, you need to map the regions via bus_space(9), and then
you can bus_space_read/bus_space_write the regions. There are multiple
examples of this is sys/dev/pci/

from userland, there's no general way of doing it; it's hard to give
advices without more details about what you want to do.

[Gnanasekar Loganathan]

Hi Manuel,

The requirement was, one of my customer want to do read/write on any
address/offset from any PCIe devices present in the system. He don't
want to use the corresponding device drivers, need separate
application that can map the PCIe device region of any devices by bus
no, device no and function. The application meant for testing purpose
only.

Using the 'pcictl' utility i can able to do read/write operations on
PCIe configuration space, but not able to do in io/mem/prefetchable
space. Able to map the region by 'bus_space_map', if the region was
not already mapped by the driver. bus_space_map fails if it was
already mapped.

Is there any way to map the region which was already mapped by the driver?

Thanks for your response.

Regards,
Gnana

Post by Manuel Bouyer

Post by Gnanasekar Loganathan
Hi Greg,
Thanks for the link, there i can found some useful information.
As we know that the 'pcictl' command will list all the pci devices
present in the system with corresponding device/vendor id.
The same way, is that possible to get/map the IO/Mem/prefetchable
region of any PCI device by using bus_space_map for Peek/Poke?

I'm not sure what you want to do. Do you want to access the device
from kernel or userland ?
From kernel, you need to map the regions via bus_space(9), and then
you can bus_space_read/bus_space_write the regions. There are multiple
examples of this is sys/dev/pci/
from userland, there's no general way of doing it; it's hard to give
advices without more details about what you want to do.
--
    NetBSD: 26 ans d'experience feront toujours la difference
--

[Manuel Bouyer]

Post by Gnanasekar Loganathan
Hi Manuel,
The requirement was, one of my customer want to do read/write on any
address/offset from any PCIe devices present in the system. He don't
want to use the corresponding device drivers, need separate
application that can map the PCIe device region of any devices by bus
no, device no and function. The application meant for testing purpose
only.
Using the 'pcictl' utility i can able to do read/write operations on
PCIe configuration space, but not able to do in io/mem/prefetchable
space. Able to map the region by 'bus_space_map', if the region was
not already mapped by the driver. bus_space_map fails if it was
already mapped.
Is there any way to map the region which was already mapped by the driver?

Well, this is not so easy.
For I/O space, I guess they can be acceeded using the i386_iopl(2) call
(assuming you're on x86 hardware). For memory-mapped space, maybe
they can be acceeded using /dev/mem, but I never tried this.

[Jonathan A. Kollasch]

Post by Manuel Bouyer

Post by Gnanasekar Loganathan
Hi Manuel,
The requirement was, one of my customer want to do read/write on any
address/offset from any PCIe devices present in the system. He don't
want to use the corresponding device drivers, need separate
application that can map the PCIe device region of any devices by bus
no, device no and function. The application meant for testing purpose
only.
Using the 'pcictl' utility i can able to do read/write operations on
PCIe configuration space, but not able to do in io/mem/prefetchable
space. Able to map the region by 'bus_space_map', if the region was
not already mapped by the driver. bus_space_map fails if it was
already mapped.
Is there any way to map the region which was already mapped by the driver?

Well, this is not so easy.
For I/O space, I guess they can be acceeded using the i386_iopl(2) call
(assuming you're on x86 hardware). For memory-mapped space, maybe
they can be acceeded using /dev/mem, but I never tried this.

Also, you can mmap(4) /dev/pci now ...

Jonathan Kollasch

[Masao Uebayashi]

Post by Jonathan A. Kollasch

Post by Manuel Bouyer

Post by Gnanasekar Loganathan
Hi Manuel,
The requirement was, one of my customer want to do read/write on any
address/offset from any PCIe devices present in the system. He don't
want to use the corresponding device drivers, need separate
application that can map the PCIe device region of any devices by bus
no, device no and function. The application meant for testing purpose
only.
Using the 'pcictl' utility i can able to do read/write operations on
PCIe configuration space, but not able to do in io/mem/prefetchable
space. Able to map the region by 'bus_space_map', if the region was
not already mapped by the driver. bus_space_map fails if it was
already mapped.
Is there any way to map the region which was already mapped by the driver?

Mappings to kernel (driver) and userspace are very different. This
is not clearly documented, but my understanding is that you can't
mmap devices that are already mapped to kernel (as unmanaged).

Post by Jonathan A. Kollasch

Post by Manuel Bouyer
Well, this is not so easy.
For I/O space, I guess they can be acceeded using the i386_iopl(2) call
(assuming you're on x86 hardware). For memory-mapped space, maybe
they can be acceeded using /dev/mem, but I never tried this.

Also, you can mmap(4) /dev/pci now ...

Yes, and it's backend is sooooo MD.

[Gnanasekar Loganathan]

Correct me if my understanding was wrong.
Conclusion is that we cannot map the PCIe regions from the separate
applicaiton which is already mapped by the driver.
The only way is the correspoding device driver should give an ioctl
command to do read/write in their address space.


Thanks,
Gnana

Post by Jonathan A. Kollasch

Post by Manuel Bouyer

Post by Gnanasekar Loganathan
Hi Manuel,
The requirement was, one of my customer want to do read/write on any
address/offset from any PCIe devices present in the system. He don't
want to use the corresponding device drivers, need separate
application that can map the PCIe device region of any devices by bus
no, device no and function. The application meant for testing purpose
only.
Using the 'pcictl' utility i can able to do read/write operations on
PCIe configuration space, but not able to do in io/mem/prefetchable
space.  Able to map the region by 'bus_space_map', if the region was
not already mapped by the driver. bus_space_map fails if it was
already mapped.
Is there any way  to map the region which was already mapped by the driver?

Mappings to kernel (driver) and userspace are very different.  This
is not clearly documented, but my understanding is that you can't
mmap devices that are already mapped to kernel (as unmanaged).

Post by Jonathan A. Kollasch

Post by Manuel Bouyer
Well, this is not so easy.
For I/O space, I guess they can be acceeded using the i386_iopl(2) call
(assuming you're on x86 hardware). For memory-mapped space, maybe
they can be acceeded using /dev/mem, but I never tried this.

Also, you can mmap(4) /dev/pci now ...

Yes, and it's backend is sooooo MD.

[David Laight]

Post by Gnanasekar Loganathan
The only way is the correspoding device driver should give an ioctl
command to do read/write in their address space.

The device driver could use a pread()/pwrite() request in order to
do the required transfer (taking an offset into the devices PCIe window).
This saves a few data copies and much faffing.

It also lets the driver use any available PCIe dma master engine to
speed up the transfer.
Single word PCIe transfers can be a slow way of copying buffers!
The DMA engine can issue a single PCIe transaction for (IIRC) 128 bytes,
for some slaves [1] these will complete almost as quickly as a single cycle.

David

[1] I've seen 200+us for a 32bit read from a ppc into an fpga.
This is ISA bus speed :-(