Question on HDLC and raw access to T1/E1 serial streams.
I am looking for a way to bridge a T1/E1 network by reading a raw bitstream from one T1 interface and writing it out to the other. The application is adding delay and/or bit corruptions for impairment testing.
I have been using Sangoma's drivers and NICs, but I'm having no luck getting their latest stuff to work so I was hoping to use in-kernel drivers (even if that means writing or hiring someone to write new ones.)
Is there currently a way to read/write the raw bitstream for a full T1 or E1 or a subset of channels?
Thanks,
Ben
Signature · 2 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com
Quoted 11 lines · Ben Greear
I am looking for a way to bridge a T1/E1 network by reading a raw bitstream from one T1 interface and writing it out to the other. The application is adding delay and/or bit corruptions for impairment testing.
I have been using Sangoma's drivers and NICs, but I'm having no luck getting their latest stuff to work so I was hoping to use in-kernel drivers (even if that means writing or hiring someone to write new ones.)
Is there currently a way to read/write the raw bitstream for a full T1 or E1 or a subset of channels?
Well, my generic HDLC works with HDLC framing only, T1/E1 is a layer lower than that... I think Cyclades have (had?) a version of PC300 card with T1/E1 interface. It at least doesn't require any "binary blobs", though I think the driver would need some work.
Which line interface do you need? G.703?
Do you need to bridge multiple streams (not slots) over one interface (internal (de)multiplexer - I mean "more than one subset of channels")?
Signature · 1 line
Krzysztof Halasa
Quoted 21 lines · Krzysztof Halasa
Quoted 5 lines · Ben Greear
I am looking for a way to bridge a T1/E1 network by reading a raw bitstream from one T1 interface and writing it out to the other. The application is adding delay and/or bit corruptions for impairment testing.
Quoted 7 lines
I have been using Sangoma's drivers and NICs, but I'm having no luck getting their latest stuff to work so I was hoping to use in-kernel drivers (even if that means writing or hiring someone to write new ones.)
Is there currently a way to read/write the raw bitstream for a full T1 or E1 or a subset of channels?
Well, my generic HDLC works with HDLC framing only, T1/E1 is a layer lower than that... I think Cyclades have (had?) a version of PC300 card with T1/E1 interface. It at least doesn't require any "binary blobs", though I think the driver would need some work.
Quoted 5 lines
Which line interface do you need? G.703?
Do you need to bridge multiple streams (not slots) over one interface (internal (de)multiplexer - I mean "more than one subset of channels")?
For protocols running HDLC as transport, I can just bridge HDLC frames. I would want to be able to select the channel(s) for the HDLC frames.
For bridging something like voice, I think if I could break out an individual channel into a bit-stream interface, I could just read bits off on one T1 channel and write to the other. Preferably, I could also bond multiple channels (including an entire T1 or E1) and bridge it as raw bits too.
I think if I could support these scenarios below, I would have everything I need:
* Configure T1 as unchannelized bitstream, bridge entire thing to second T1.
* Configure channels 1-5 as a bitstream and bridge that to channels 1-5 of a second T1. (random proprietary bit-streaming protocol, would probably bridge HDLC just fine, but handling
HDLC as frames would be more efficient I think.)
channels 6-10 configured as an HDLC interface, bridged as HDLC
frames to channels 6-10 of a second T1. (PPP & other protocols over HDLC)
channels 10-24 each configured as a separate bit-stream, bridged to
channels 10-24 on the second T1. (Voice)
* Configure entire T1 as HDLC transport, bridge HDLC frames from one T1 to the other.
Does that make sense?
Thanks,
Ben
Signature · 1 line
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com
Quoted 5 lines · Ben Greear
I think if I could support these scenarios below, I would have everything I need:
* Configure T1 as unchannelized bitstream, bridge entire thing to second T1.
I think it should be easy with such card, though I think the drivers can't currently do that.
Quoted 3 lines
* Configure channels 1-5 as a bitstream and bridge that to channels 1-5
of a second T1. (random proprietary bit-streaming protocol,
I think the hardware would permit that. Probably needs additional driver support and I'm not sure about timeslot synchronization (for HDLC, sync doesn't matter).
Quoted 2 lines
would probably bridge HDLC just fine, but handling
HDLC as frames would be more efficient I think.)
Not sure, maybe yes (less data to bridge due to bit stuffing, flags etc.), maybe not (variable length of HDLC frame).
Quoted 5 lines
channels 6-10 configured as an HDLC interface, bridged as HDLC
frames to channels 6-10 of a second T1. (PPP & other protocols over HDLC)
channels 10-24 each configured as a separate bit-stream, bridged to
channels 10-24 on the second T1. (Voice)
I think the above could be a problem - I think common T1/E1 cards can do only one stream at once.
I wonder if it can be done in software - the hardware framer would have to pass all data transparently, and it would be demultiplexed, processed and then multiplexed by the driver. Quite complicated, but I think at 2 Mbps it wouldn't be a CPU performance problem.
Quoted 2 lines
* Configure entire T1 as HDLC transport, bridge HDLC frames from one T1 to the other.
Easy even with existing drivers I think (no bridge support but it's trivial).
Signature · 1 line
Krzysztof Halasa
Quoted 6 lines · Krzysztof Halasa
Quoted 3 lines · Ben Greear
I think if I could support these scenarios below, I would have everything I need:
Quoted 7 lines
Quoted 3 lines
* Configure T1 as unchannelized bitstream, bridge entire thing to second T1.
I think it should be easy with such card, though I think the drivers can't currently do that.
At least some out-of-tree drivers seem to be able to do this. For instance, these guys:
http://www.farsite.co.uk/
Quoted 9 lines
Quoted 4 lines
* Configure channels 1-5 as a bitstream and bridge that to channels 1-5
of a second T1. (random proprietary bit-streaming protocol,I think the hardware would permit that. Probably needs additional driver support and I'm not sure about timeslot synchronization (for HDLC, sync doesn't matter).
My assumption for bridging a bitstream is that timeslot sync is not absolutely critical. However, IF you could be sure of time-slot sync, you'd have a lot more power and be able to do some extra ticks in user-space I think...
Quoted 7 lines
Quoted 3 lines
would probably bridge HDLC just fine, but handling
HDLC as frames would be more efficient I think.)Not sure, maybe yes (less data to bridge due to bit stuffing, flags etc.), maybe not (variable length of HDLC frame).
The key for me is that if you ever miss a slot in bit-stream mode, you can never make it up because every bit is critical. This leads to having to drop arbitrary data to keep from ever-increasing latency on your bridge. With HDLC, you can skip the flags and make up time if you ever miss a timeslot (assuming the HDLC is not using the line at 100% capacity.)
Quoted 7 lines
Quoted 6 lines
channels 6-10 configured as an HDLC interface, bridged as HDLC
frames to channels 6-10 of a second T1. (PPP & other protocols over HDLC)
channels 10-24 each configured as a separate bit-stream, bridged to
channels 10-24 on the second T1. (Voice)
Quoted 8 lines
I think the above could be a problem - I think common T1/E1 cards can do only one stream at once.
I wonder if it can be done in software - the hardware framer would have to pass all data transparently, and it would be demultiplexed, processed and then multiplexed by the driver. Quite complicated, but I think at 2 Mbps it wouldn't be a CPU performance problem.
I'd be happy with a software approach. In fact, if I could get a framed packet (ie, I know that byte 0 is channel 1, byte 24 is channel 24, and byte 25 is channel 1 again...) then I could even do the multiplexing in user space.
For write, I'd also need to be able to guarantee that byte 0 goes to channel 1, etc. So, if the driver bit-stuffed, then it would need to do an entire time-slice at once.
Quoted 7 lines
Quoted 3 lines
* Configure entire T1 as HDLC transport, bridge HDLC frames from one T1 to the other.
Easy even with existing drivers I think (no bridge support but it's trivial).
Excellent. I actually want to write the bridge logic myself in user-space..I just need the driver API and at least one driver that supports it and has support for readily available T1/E1 hardware. For future work, I'd be interested in the same sort of support for DS3/E3.
I'd be willing to fund development of these features if you or someone else is interested (please contact me off list for details.)
Thanks,
Ben
Signature · 1 line
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com
Ar Sul, 2006-10-01 am 11:41 -0700, ysgrifennodd Ben Greear:
Quoted 3 lines
At least some out-of-tree drivers seem to be able to do this. For instance, these guys:
http://www.farsite.co.uk/
The crazy Zaptel guys can probably do precisely what you need as their hardware though aimed at voice work lets software get in at an extremely low level.
Quoted 11 lines · Alan Cox
Ar Sul, 2006-10-01 am 11:41 -0700, ysgrifennodd Ben Greear:
Quoted 3 lines
At least some out-of-tree drivers seem to be able to do this. For instance, these guys:
http://www.farsite.co.uk/The crazy Zaptel guys can probably do precisely what you need as their hardware though aimed at voice work lets software get in at an extremely low level.
I think you might be right. It will take some more experimenting to see if they allow enough control to read/write bitstreams (clear channel groups in their terminology, I believe) while at the same time keeping the timing slots in order...
Thanks,
Ben
Quoted 6 lines
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 1 line
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com
Quoted 5 lines · Ben Greear
I think you might be right. It will take some more experimenting to see if they allow
enough control to read/write bitstreams (clear channel groups in their terminology, I believe) while at
the same time keeping the timing slots in order...
Sangoma also makes cards with linux drivers, which I suspect will let you at the low levels if you want to. They certainly support zaptel stuff on their cards. Their hardware is also quite a bit cheaper than farsite's in my experience.
--
Len Sorensen
Quoted 7 lines · Lennart Sorensen
Quoted 4 lines · Ben Greear
I think you might be right. It will take some more experimenting to see if they allow enough control to read/write bitstreams (clear channel groups in their terminology, I believe) while at the same time keeping the timing slots in order...
Quoted 5 lines
Sangoma also makes cards with linux drivers, which I suspect will let you at the low levels if you want to. They certainly support zaptel stuff on their cards. Their hardware is also quite a bit cheaper than farsite's in my experience.
Sangoma's NICs used to work for me..but it seems their latest drivers broke the support that I need. I think there aren't too many people interested in bit-streaming, so the feature probably doesn't get tested that often... If I can get the digium hardware to work with a particular zaptel.conf, then I can try Sangoma and see if it will work when configured in that same manner...
Thanks,
Ben
Quoted 8 lines
--
Len Sorensen
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 1 line
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com
Quoted 5 lines · Ben Greear
My assumption for bridging a bitstream is that timeslot sync is not absolutely critical. However, IF
you could be sure of time-slot sync, you'd have a lot more power and be able to do some extra ticks
in user-space I think...
Not sure what do you want to do with that, but it may be critical for many things.
Quoted 3 lines
The key for me is that if you ever miss a slot in bit-stream mode, you can never make it up because
every bit is critical.
I think you'd have to perform some recover procedure then, that's similar to DCE losing sync.
Quoted 3 lines
This leads to having to drop arbitrary data to
keep from ever-increasing latency on your
bridge.
If your clocks are synchronized (for example, if you get a "master" clock from your public phone exchange and you propagate it downstream, or your machine is the "master") then you never drop anything, the input and output rates are equal and in sync.
Quoted 3 lines
With HDLC, you can skip the flags and make up time if you
ever miss a timeslot (assuming the
HDLC is not using the line at 100% capacity.)
Sure. Even at 100% you can just drop a frame, HDLC applications must be prepared for it.
Quoted 5 lines
I'd be happy with a software approach. In fact, if I could get a framed packet (ie, I know that
byte 0 is channel 1, byte 24 is channel 24, and byte 25 is channel 1 again...) then I could even
do the multiplexing in user space.
Well, with software framing it would look a bit different - there is no such thing as "packet" as both TX and RX is continuous, not aligned to anything etc. You would have to detect channel boundaries, and bit-shift the data. Requires the sync serial controller (and FALC) in transparent mode (I would have to look at some docs). I think the kernel is a better place for things like that due to latency issues.
Quoted 3 lines
For write, I'd also need to be able to guarantee that byte 0 goes to channel 1, etc. So, if the
driver bit-stuffed, then it would need to do an entire time-slice at once.
BTW: An HDLC frame can use many slices. You can in fact have many HDLC frames (from different streams) multiplexed. You just need a multi-stream device or a multiplexer.
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
* Configure entire T1 as HDLC transport, bridge HDLC frames from one T1 to the other.
Excellent. I actually want to write the bridge logic myself in user-space..I just need the driver
API and at least one driver that supports it and has support for readily available T1/E1 hardware.
If you want the userspace HDLC bridge... I'd use a pair of T1/E1 cards with generic HDLC support, for example, Cyclades PC300 (never used them and while I don't exactly like their driver, in case of problems I could add T1/E1 to my own driver which currently supports PC300 X.21 and V.24/V.35).
Once T1/E1s are working and the required slots are selected:
sethdlc hdlc0 hdlc (options)
sethdlc hdlc1 hdlc (options)
ifconfig hdlc0 up
ifconfig hdlc1 up
man PF_PACKET
A single HDLC stream is a simple thing because it's exactly what the cards are designed for.
Signature · 1 line
Krzysztof Halasa
Quoted 14 lines · Krzysztof Halasa
Quoted 6 lines · Ben Greear
My assumption for bridging a bitstream is that timeslot sync is not absolutely critical. However, IF
you could be sure of time-slot sync, you'd have a lot more power and be able to do some extra ticks
in user-space I think...Not sure what do you want to do with that, but it may be critical for many things.
Quoted 8 lines
Quoted 4 lines
The key for me is that if you ever miss a slot in bit-stream mode, you can never make it up because
every bit is critical.I think you'd have to perform some recover procedure then, that's similar to DCE losing sync.
The recovery is to simply drop a few time slices. Whatever protocol is running over the wire must be able to deal with it.
Quoted 10 lines
Quoted 4 lines
This leads to having to drop arbitrary data to
keep from ever-increasing latency on your
bridge.If your clocks are synchronized (for example, if you get a "master" clock from your public phone exchange and you propagate it downstream, or your machine is the "master") then you never drop anything, the input and output rates are equal and in sync.
If you are bridging in user-space (or kernel, for that matter), it is possible your writing process will not be scheduled in time to fill up the tx buffer with real data. So, it would have to be padded by the driver and/or NIC hardware. With an adequately powered machine, this should be a rare occurrence, however.
Quoted 8 lines
Quoted 4 lines
With HDLC, you can skip the flags and make up time if you
ever miss a timeslot (assuming the
HDLC is not using the line at 100% capacity.)Sure. Even at 100% you can just drop a frame, HDLC applications must be prepared for it.
Agreed.
Quoted 7 lines
Quoted 6 lines
I'd be happy with a software approach. In fact, if I could get a framed packet (ie, I know that
byte 0 is channel 1, byte 24 is channel 24, and byte 25 is channel 1 again...) then I could even
do the multiplexing in user space.
Quoted 8 lines
Well, with software framing it would look a bit different - there is no such thing as "packet" as both TX and RX is continuous, not aligned to anything etc. You would have to detect channel boundaries, and bit-shift the data. Requires the sync serial controller (and FALC) in transparent mode (I would have to look at some docs). I think the kernel is a better place for things like that due to latency issues.
I would definitely want the driver/hardware to deal with the byte/bit boundaries. But, the user-space read API could be similar to UDP (or HDLC if I understand it correctly), but it would always return a multiple of the number of channels in bytes (0, 24, 48, ... for and entire T1).
Quoted 9 lines
Quoted 4 lines
For write, I'd also need to be able to guarantee that byte 0 goes to channel 1, etc. So, if the
driver bit-stuffed, then it would need to do an entire time-slice at once.BTW: An HDLC frame can use many slices. You can in fact have many HDLC frames (from different streams) multiplexed. You just need a multi-stream device or a multiplexer.
Good.
Quoted 22 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
* Configure entire T1 as HDLC transport, bridge HDLC frames from one T1 to the other.
Excellent. I actually want to write the bridge logic myself in user-space..I just need the driver
API and at least one driver that supports it and has support for readily available T1/E1 hardware.If you want the userspace HDLC bridge... I'd use a pair of T1/E1 cards with generic HDLC support, for example, Cyclades PC300 (never used them and while I don't exactly like their driver, in case of problems I could add T1/E1 to my own driver which currently supports PC300 X.21 and V.24/V.35).
Once T1/E1s are working and the required slots are selected:
sethdlc hdlc0 hdlc (options)
sethdlc hdlc1 hdlc (options)
ifconfig hdlc0 up
ifconfig hdlc1 up
manPF_PACKET
Quoted 3 lines
A single HDLC stream is a simple thing because it's exactly what the cards are designed for.
Before I settle on hardware, I'd like to have some idea that the NIC can deal with raw bit-streams. It seems all of the NICs can handle HDLC, so that part should be pretty easy.
Thanks,
Ben
Signature · 1 line
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com
Quoted 4 lines · Ben Greear
I would definitely want the driver/hardware to deal with the byte/bit boundaries. But,
the user-space read API could be similar to UDP (or HDLC if I understand it correctly),
I think so.
Quoted 3 lines
Before I settle on hardware, I'd like to have some idea that the NIC can deal with raw
bit-streams.
I will look at FALC docs. Most (?) sync serial processors can RX/TX transparent data (though for E3 rates it would be better to have some hardware support).
Signature · 1 line
Krzysztof Halasa