Skip to contentSkip to conversation

SAL PCI config space

DISCUSSION Matthew Wilcox 5 participants 10 messages Apr 30, 2004

lore
Share & tools
lore
#0 lore raw

ia64_sal_pci_config_read() and ia64_sal_pci_config_write() use SAL_CALL(). This takes an IRQ-safe spinlock, but SAL 3.2 says that SAL_PCI_CONFIG_READ and SAL_PCI_CONFIG_WRITE are both MP-safe and reentrant. So is there any reason we shouldn't use SAL_CALL_REENTRANT?

(at the moment, we're protected by pci_lock so it wouldn't matter, but I'm just about to submit a patch to do away with that)

Signature · 6 lines
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain
#1 lore raw
Quoted 5 lines · Matthew Wilcox

ia64_sal_pci_config_read() and ia64_sal_pci_config_write() use SAL_CALL(). This takes an IRQ-safe spinlock, but SAL 3.2 says that SAL_PCI_CONFIG_READ and SAL_PCI_CONFIG_WRITE are both MP-safe and reentrant. So is there any reason we shouldn't use SAL_CALL_REENTRANT?

I thought there was going to be further clarification on this, but
I'm not sure what SAL rev it would be in. On HP boxes, we can't preform multiple config accesses to the same root bus at the same time. So far, the SAL_CALL lock has protected us from needing to worry about it. I'd suspect other boxes might have similar restrictions.

Alex

Quoted 2 lines

(at the moment, we're protected by pci_lock so it wouldn't matter, but I'm just about to submit a patch to do away with that)

#2 lore raw
In reply to Matthew Wilcox
Quoted 4 lines

ia64_sal_pci_config_read() and ia64_sal_pci_config_write() use SAL_CALL(). This takes an IRQ-safe spinlock, but SAL 3.2 says that SAL_PCI_CONFIG_READ and SAL_PCI_CONFIG_WRITE are both MP-safe and reentrant. So is there any reason we shouldn't use SAL_CALL_REENTRANT?

Theoretically there might be older SAL implementations where these aren't re-entrant ... so to be really safe, you'd have to check that the SAL version is >= 3.2 before using the SAL_CALL_REENTRANT form.

-Tony

#3 lore raw
In reply to Matthew Wilcox
Quoted 8 lines · Tony
Quoted 4 lines

ia64_sal_pci_config_read() and ia64_sal_pci_config_write() use SAL_CALL(). This takes an IRQ-safe spinlock, but SAL 3.2 says that SAL_PCI_CONFIG_READ and SAL_PCI_CONFIG_WRITE are both MP-safe and reentrant. So is there any reason we shouldn't use SAL_CALL_REENTRANT?

Theoretically there might be older SAL implementations where these aren't re-entrant ... so to be really safe, you'd have to check that the SAL version is >= 3.2 before using the SAL_CALL_REENTRANT form.

Gack. I went and dug up some older SAL specs.

SAL 3.1 is ambiguous. On page 9-4, it says "Yes" under "Re-entrancy requirement" for these calls, but then on 9-18 and 9-19, it says, "Good programming practices dictate that indexed accesses to the configuration space be serialized in order to be MP-safe."

(this seems rather looney to me. how can you have something that's reentrant but not MP-safe? I think it's an unintended carry-over from SAL 3.0)

SAL 3.0 states "SAL procedures are not re-entrant with respect to any runtime service (including itself)."

So what's actually shipping in systems? David's against runtime checks due to some of the buggy firmware hp's managed to ship. I agree with him that it would have been much more sensible had the SAL spec specifid new calls rather than changing the semantics of existing calls.

Signature · 6 lines
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain
#4 lore raw
In reply to Matthew Wilcox
Quoted 4 lines

So what's actually shipping in systems? David's against runtime checks due to some of the buggy firmware hp's managed to ship. I agree with him that it would have been much more sensible had the SAL spec specifid new calls rather than changing the semantics of existing

calls.

It's not whats shipping that's the big problem ... it's the people still running BigSur, Lion and other prototype systems that will never have their firmware upgraded to SAL3.2 spec.

Apart from general cleanliness of not having functions that are re-entrant and MP safe uselessly spin on a lock to serialize, do you see any tangible benefits from switching these to use the SAL_CALL_REENTRANT method? Surely config space accesses aren't in anyone's critical path?

-Tony

#5 lore raw
In reply to Matthew Wilcox

Most of the SAL calls except the pure reading ones w/o affecting any state have to be reentrant and MP safe, don't they? Multiple OSs/executables on different Soft partitions do not share others' lock variables, unless otherwise all SAL calls of soft partitions are redirected to one hosting OS, which sounds not reasonable.
So, SAL_CALL_REENTRANT could be a solution for the kernel itself, but not for partitioned platforms.

Thanks,
John

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Luck, Tony
Sent: Friday, April 30, 2004 1:09 PM
To: Matthew Wilcox
Cc: [email protected]
Subject: RE: SAL PCI config space

Quoted 4 lines

So what's actually shipping in systems? David's against runtime checks due to some of the buggy firmware hp's managed to ship. I agree with him that it would have been much more sensible had the SAL spec specifid new calls rather than changing the semantics of existing

calls.

It's not whats shipping that's the big problem ... it's the people still running BigSur, Lion and other prototype systems that will never have their firmware upgraded to SAL3.2 spec.

Apart from general cleanliness of not having functions that are re-entrant and MP safe uselessly spin on a lock to serialize, do you see any tangible benefits from switching these to use the SAL_CALL_REENTRANT method? Surely config space accesses aren't in anyone's critical path?

-Tony
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#6 lore raw
In reply to Matthew Wilcox
Quoted 6 lines · John Lee

Most of the SAL calls except the pure reading ones w/o affecting any state have to be reentrant and MP safe, don't they? Multiple OSs/executables on different Soft partitions do not share others' lock variables, unless otherwise all SAL calls of soft partitions are redirected to one hosting OS, which sounds not reasonable.

Yes. I think that's why the SAL spec changed because all the vendors want to support virtual machines.
But that doesn't help the older boxes which do not implement the newer SAL spec or have "non-compliant" implementations.

Quoted 2 lines

So, SAL_CALL_REENTRANT could be a solution for the kernel itself, but not for partitioned platforms.

Sorry - this statement doesn't make sense to me. You meant s/SAL_CALL_REENTRANT/spinlocks/ above maybe?

grant

Quoted 40 lines

Thanks,
John

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Luck, Tony
Sent: Friday, April 30, 2004 1:09 PM
To: Matthew Wilcox
Cc: [email protected]
Subject: RE: SAL PCI config space

Quoted 4 lines

So what's actually shipping in systems? David's against runtime checks due to some of the buggy firmware hp's managed to ship. I agree with him that it would have been much more sensible had the SAL spec specifid new calls rather than changing the semantics of existing

calls.

It's not whats shipping that's the big problem ... it's the people still running BigSur, Lion and other prototype systems that will never have their firmware upgraded to SAL3.2 spec.

Apart from general cleanliness of not having functions that are re-entrant and MP safe uselessly spin on a lock to serialize, do you see any tangible benefits from switching these to use the SAL_CALL_REENTRANT method? Surely config space accesses aren't in anyone's critical path?

(quoted diff) +0 -2
-Tony
-

To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#7 lore raw
In reply to Matthew Wilcox
Quoted 9 lines
Quoted 4 lines

Multiple OSs/executables on different Soft partitions do not share others' lock variables, unless otherwise all SAL calls of soft partitions are redirected to one hosting OS, which sounds not reasonable.

Yes. I think that's why the SAL spec changed because all the vendors want to support virtual machines.
But that doesn't help the older boxes which do not implement the newer SAL spec or have "non-compliant" implementations.

There would be other issues besides whether some SAL calls are re-entrant to running multiple OS in soft partitions. E.g. MCA

-Tony

#8 lore raw
In reply to Matthew Wilcox
Quoted 16 lines

-----Original Message-----
From: Grant Grundler [mailto:[email protected]]
Sent: Friday, April 30, 2004 2:46 PM
To: John Lee
Cc: Luck, Tony; Matthew Wilcox; [email protected]
Subject: Re: SAL PCI config space

Quoted 6 lines · John Lee

Most of the SAL calls except the pure reading ones w/o affecting any state have to be reentrant and MP safe, don't they? Multiple OSs/executables on different Soft partitions do not share others' lock variables, unless otherwise all SAL calls of soft partitions are redirected to one hosting OS, which sounds not reasonable.

Quoted 2 lines

Yes. I think that's why the SAL spec changed because all the vendors want to support virtual machines.

Right. So it's SAL implementers' responsibility to make them mp-safe.

Quoted 11 lines

But that doesn't help the older boxes which do not implement the newer SAL spec or have "non-compliant" implementations.

Quoted 1 line

So, SAL_CALL_REENTRANT could be a solution for the kernel

itself, but

Quoted 1 line

not for partitioned platforms.

Sorry - this statement doesn't make sense to me. You meant s/SAL_CALL_REENTRANT/spinlocks/ above maybe?

grant

For Linux's own backward compatibility with older machines, I'm with Tony - SAL_CALL_REENTRANT -, until we all agree not to worry about them.

Thanks,
John

#9 lore raw
In reply to Matthew Wilcox
Quoted 13 lines

-----Original Message-----
From: Luck, Tony [mailto:[email protected]]
Sent: Friday, April 30, 2004 2:59 PM
To: Grant Grundler; John Lee
Cc: Matthew Wilcox; [email protected]
Subject: RE: SAL PCI config space

Quoted 5 lines
Quoted 4 lines

Multiple OSs/executables on different Soft partitions do not share others' lock variables, unless otherwise all SAL calls of soft partitions are redirected to one hosting OS, which sounds not reasonable.

Quoted 8 lines
Quoted 4 lines

Yes. I think that's why the SAL spec changed because all the vendors want to support virtual machines.
But that doesn't help the older boxes which do not implement the newer SAL spec or have "non-compliant" implementations.

There would be other issues besides whether some SAL calls are re-entrant to running multiple OS in soft partitions. E.g. MCA

I'm already there. RAS for soft partitions ;)
Any forum or discussion on that front would be helpful if you could invite us.

Thanks,
John Lee