PCI: dwc: Fix BAR0 wrong map to iATU6 after root complex reinit endpoint
Subsystems (4)
Frank Li
maintainerPATCH 1/1 PCI: dwc: Fix BAR0 wrong map to iATU6 after root complex reinit endpoint
dw_pcie_ep_inbound_atu()
{
...
if (!ep->bar_to_atu[bar])
free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows);
else
free_win = ep->bar_to_atu[bar];
...
}The atu index 0 is valid case for atu number. The find_first_zero_bit() will return 6 when second time call into this function if atu is 0. Suppose it should use branch 'free_win = ep->bar_to_atu[bar]'.
Change 'bar_to_atu' to s8. Initialize bar_to_atu as -1 to indicate it have not allocate atu to the bar.
Close: https://lore.kernel.org/linux-pci/ZXt2A+Fusfz3luQV@x1-carbon/T/#u
---
Notes:
@Niklas:
I have not test your case. I should be equal to previous's fix in
mail list.
drivers/pci/controller/dwc/pcie-designware-ep.c | 11 ++++++++--- drivers/pci/controller/dwc/pcie-designware.h | 2 +- 2 files changed, 9 insertions(+), 4 deletions(-)
drivers/pci/controller/dwc/pcie-designware-ep.c +8 -3
@@ -174,7 +174,7 @@ static int dw_pcie_ep_inbound_atu(struct dw_pcie_ep *ep, u8 func_no, int type,u32 free_win;struct dw_pcie *pci =to_dw_pcie_from_ep (ep); -if (!ep->bar_to_atu[bar]) +if (ep->bar_to_atu[bar] <0 ) free_win =find_first_zero_bit (ep->ib_window_map, pci->num_ib_windows);else free_win = ep->bar_to_atu[bar];
@@ -228,14 +228,17 @@ static void dw_pcie_ep_clear_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,struct dw_pcie_ep *ep =epc_get_drvdata (epc);struct dw_pcie *pci =to_dw_pcie_from_ep (ep);enum pci_barno bar = epf_bar->barno; -u32 atu_index = ep->bar_to_atu[bar]; +s8 atu_index = ep->bar_to_atu[bar]; + +if (atu_index <0 ) +return ;__dw_pcie_ep_reset_bar (pci, func_no, bar, epf_bar->flags);dw_pcie_disable_atu (pci,PCIE_ATU_REGION_DIR_IB , atu_index);clear_bit (atu_index, ep->ib_window_map); ep->epf_bar[bar] =NULL ; - ep->bar_to_atu[bar] =0 ; + ep->bar_to_atu[bar] =-1 ; } static int dw_pcie_ep_set_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
@@ -767,6 +770,8 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)return -ENOMEM ; ep->outbound_addr = addr; +memset (ep->bar_to_atu,-1 ,sizeof (ep->bar_to_atu)); + epc =devm_pci_epc_create (dev, &epc_ops);if (IS_ERR (epc)) {dev_err (dev,"Failed to create epc device\n" );
drivers/pci/controller/dwc/pcie-designware.h +1 -1
@@ -362,7 +362,7 @@ struct dw_pcie_ep {phys_addr_t phys_base;size_t addr_size;size_t page_size; -u8 bar_to_atu[PCI_STD_NUM_BARS ]; +s8 bar_to_atu[PCI_STD_NUM_BARS ];phys_addr_t *outbound_addr;unsigned long *ib_window_map;unsigned long *ob_window_map;
Signature · 1 line
2.34.1
Frank Li
maintainerPATCH 1/1 PCI: dwc: Fix BAR0 wrong map to iATU6 after root complex reinit endpoint
dw_pcie_ep_inbound_atu()
{
...
if (!ep->bar_to_atu[bar])
free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows);
else
free_win = ep->bar_to_atu[bar];
...
}The atu index 0 is valid case for atu number. The find_first_zero_bit() will return 6 when second time call into this function if atu is 0. Suppose it should use branch 'free_win = ep->bar_to_atu[bar]'.
Change 'bar_to_atu' to s8. Initialize bar_to_atu as -1 to indicate it have not allocate atu to the bar.
Close: https://lore.kernel.org/linux-pci/ZXt2A+Fusfz3luQV@x1-carbon/T/#u
---
Notes:
@Niklas:
I have not test your case. I should be equal to previous's fix in
mail list.
drivers/pci/controller/dwc/pcie-designware-ep.c | 11 ++++++++--- drivers/pci/controller/dwc/pcie-designware.h | 2 +- 2 files changed, 9 insertions(+), 4 deletions(-)
drivers/pci/controller/dwc/pcie-designware-ep.c +8 -3
@@ -174,7 +174,7 @@ static int dw_pcie_ep_inbound_atu(struct dw_pcie_ep *ep, u8 func_no, int type,u32 free_win;struct dw_pcie *pci =to_dw_pcie_from_ep (ep); -if (!ep->bar_to_atu[bar]) +if (ep->bar_to_atu[bar] <0 ) free_win =find_first_zero_bit (ep->ib_window_map, pci->num_ib_windows);else free_win = ep->bar_to_atu[bar];
@@ -228,14 +228,17 @@ static void dw_pcie_ep_clear_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,struct dw_pcie_ep *ep =epc_get_drvdata (epc);struct dw_pcie *pci =to_dw_pcie_from_ep (ep);enum pci_barno bar = epf_bar->barno; -u32 atu_index = ep->bar_to_atu[bar]; +s8 atu_index = ep->bar_to_atu[bar]; + +if (atu_index <0 ) +return ;__dw_pcie_ep_reset_bar (pci, func_no, bar, epf_bar->flags);dw_pcie_disable_atu (pci,PCIE_ATU_REGION_DIR_IB , atu_index);clear_bit (atu_index, ep->ib_window_map); ep->epf_bar[bar] =NULL ; - ep->bar_to_atu[bar] =0 ; + ep->bar_to_atu[bar] =-1 ; } static int dw_pcie_ep_set_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
@@ -767,6 +770,8 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)return -ENOMEM ; ep->outbound_addr = addr; +memset (ep->bar_to_atu,-1 ,sizeof (ep->bar_to_atu)); + epc =devm_pci_epc_create (dev, &epc_ops);if (IS_ERR (epc)) {dev_err (dev,"Failed to create epc device\n" );
drivers/pci/controller/dwc/pcie-designware.h +1 -1
@@ -362,7 +362,7 @@ struct dw_pcie_ep {phys_addr_t phys_base;size_t addr_size;size_t page_size; -u8 bar_to_atu[PCI_STD_NUM_BARS ]; +s8 bar_to_atu[PCI_STD_NUM_BARS ];phys_addr_t *outbound_addr;unsigned long *ib_window_map;unsigned long *ob_window_map;
Signature · 1 line
2.34.1
Quoted 19 lines · Frank Li
dw_pcie_ep_inbound_atu(){ ... if (!ep->bar_to_atu[bar]) free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows); else free_win = ep->bar_to_atu[bar]; ... }The atu index 0 is valid case for atu number. The
find_first_zero_bit()will return 6 when second time call into this function if atu is 0. Suppose it should use branch 'free_win= ep->bar_to_atu[bar]'.Change '
bar_to_atu' to s8. Initializebar_to_atuas -1 to indicate it have not allocate atu to the bar.Reported-by Niklas Cassel <[email protected]>Close: https://lore.kernel.org/linux-pci/ZXt2A+Fusfz3luQV@x1-carbon/T/#u
Quoted 8 lines
Fixes 4284c88fff0e ("PCI: designware-ep: Allow pci_epc_set_bar() update inbound map address")Signed-off-by Frank Li <[email protected]>---
Notes:
@Niklas:
I have not test your case. I should be equal to previous's fix in
mail list.
Hello Frank,
Thank you for sending a proper fix for this!
Personally, I slightly prefer your fix that saves the iatu index + 1, and keeps 0 to mean unused. That way, you don't need the memset, and you don't need to change the type to signed, but either way is fine by me, so:
Quoted 63 lines
drivers/pci/controller/dwc/pcie-designware-ep.c | 11 ++++++++--- drivers/pci/controller/dwc/pcie-designware.h | 2 +- 2 files changed, 9 insertions(+), 4 deletions(-)drivers/pci/controller/dwc/pcie-designware-ep.c +8
-3@@ -174,7 +174,7 @@ static int dw_pcie_ep_inbound_atu(struct dw_pcie_ep *ep, u8 func_no, int type,u32 free_win;struct dw_pcie *pci =to_dw_pcie_from_ep (ep); -if (!ep->bar_to_atu[bar]) +if (ep->bar_to_atu[bar] <0 ) free_win =find_first_zero_bit (ep->ib_window_map, pci->num_ib_windows);else free_win = ep->bar_to_atu[bar];@@ -228,14 +228,17 @@ static void dw_pcie_ep_clear_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,struct dw_pcie_ep *ep =epc_get_drvdata (epc);struct dw_pcie *pci =to_dw_pcie_from_ep (ep);enum pci_barno bar = epf_bar->barno; -u32 atu_index = ep->bar_to_atu[bar]; +s8 atu_index = ep->bar_to_atu[bar]; + +if (atu_index <0 ) +return ;__dw_pcie_ep_reset_bar (pci, func_no, bar, epf_bar->flags);dw_pcie_disable_atu (pci,PCIE_ATU_REGION_DIR_IB , atu_index);clear_bit (atu_index, ep->ib_window_map); ep->epf_bar[bar] =NULL ; - ep->bar_to_atu[bar] =0 ; + ep->bar_to_atu[bar] =-1 ; } static int dw_pcie_ep_set_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,@@ -767,6 +770,8 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)return -ENOMEM ; ep->outbound_addr = addr; +memset (ep->bar_to_atu,-1 ,sizeof (ep->bar_to_atu)); + epc =devm_pci_epc_create (dev, &epc_ops);if (IS_ERR (epc)) {dev_err (dev,"Failed to create epc device\n" );drivers/pci/controller/dwc/pcie-designware.h +1
-1@@ -362,7 +362,7 @@ struct dw_pcie_ep {phys_addr_t phys_base;size_t addr_size;size_t page_size; -u8 bar_to_atu[PCI_STD_NUM_BARS ]; +s8 bar_to_atu[PCI_STD_NUM_BARS ];phys_addr_t *outbound_addr;unsigned long *ib_window_map;unsigned long *ob_window_map;Signature · 1 line
2.34.1
Frank Li
maintainerQuoted 29 lines · Niklas Cassel
Quoted 19 lines · Frank Li
dw_pcie_ep_inbound_atu(){ ... if (!ep->bar_to_atu[bar]) free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows); else free_win = ep->bar_to_atu[bar]; ... }The atu index 0 is valid case for atu number. The
find_first_zero_bit()will return 6 when second time call into this function if atu is 0. Suppose it should use branch 'free_win= ep->bar_to_atu[bar]'.Change '
bar_to_atu' to s8. Initializebar_to_atuas -1 to indicate it have not allocate atu to the bar.Reported-by Niklas Cassel <[email protected]>Close: https://lore.kernel.org/linux-pci/ZXt2A+Fusfz3luQV@x1-carbon/T/#u
Quoted 8 lines
Fixes 4284c88fff0e ("PCI: designware-ep: Allow pci_epc_set_bar() update inbound map address")Signed-off-by Frank Li <[email protected]>---
Notes:
@Niklas:
I have not test your case. I should be equal to previous's fix in
mail list.
Quoted 7 lines
Hello Frank,
Thank you for sending a proper fix for this!
Personally, I slightly prefer your fix that saves the iatu index + 1, and keeps 0 to mean unused. That way, you don't need the memset, and you don't need to change the type to signed, but either way is fine by me, so:
index + 1 don't match hardware iATU index. It will be confused because other parts is 0 based.
So I choose "-1" as free iATU.
Frank
Quoted 66 lines
Reviewed-by Niklas Cassel <[email protected]>Quoted 63 lines
drivers/pci/controller/dwc/pcie-designware-ep.c | 11 ++++++++--- drivers/pci/controller/dwc/pcie-designware.h | 2 +- 2 files changed, 9 insertions(+), 4 deletions(-)drivers/pci/controller/dwc/pcie-designware-ep.c +8
-3@@ -174,7 +174,7 @@ static int dw_pcie_ep_inbound_atu(struct dw_pcie_ep *ep, u8 func_no, int type,u32 free_win;struct dw_pcie *pci =to_dw_pcie_from_ep (ep); -if (!ep->bar_to_atu[bar]) +if (ep->bar_to_atu[bar] <0 ) free_win =find_first_zero_bit (ep->ib_window_map, pci->num_ib_windows);else free_win = ep->bar_to_atu[bar];@@ -228,14 +228,17 @@ static void dw_pcie_ep_clear_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,struct dw_pcie_ep *ep =epc_get_drvdata (epc);struct dw_pcie *pci =to_dw_pcie_from_ep (ep);enum pci_barno bar = epf_bar->barno; -u32 atu_index = ep->bar_to_atu[bar]; +s8 atu_index = ep->bar_to_atu[bar]; + +if (atu_index <0 ) +return ;__dw_pcie_ep_reset_bar (pci, func_no, bar, epf_bar->flags);dw_pcie_disable_atu (pci,PCIE_ATU_REGION_DIR_IB , atu_index);clear_bit (atu_index, ep->ib_window_map); ep->epf_bar[bar] =NULL ; - ep->bar_to_atu[bar] =0 ; + ep->bar_to_atu[bar] =-1 ; } static int dw_pcie_ep_set_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,@@ -767,6 +770,8 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)return -ENOMEM ; ep->outbound_addr = addr; +memset (ep->bar_to_atu,-1 ,sizeof (ep->bar_to_atu)); + epc =devm_pci_epc_create (dev, &epc_ops);if (IS_ERR (epc)) {dev_err (dev,"Failed to create epc device\n" );drivers/pci/controller/dwc/pcie-designware.h +1
-1@@ -362,7 +362,7 @@ struct dw_pcie_ep {phys_addr_t phys_base;size_t addr_size;size_t page_size; -u8 bar_to_atu[PCI_STD_NUM_BARS ]; +s8 bar_to_atu[PCI_STD_NUM_BARS ];phys_addr_t *outbound_addr;unsigned long *ib_window_map;unsigned long *ob_window_map;Signature · 1 line
2.34.1
Quoted 34 lines · Frank Li
Quoted 33 lines · Niklas Cassel
Quoted 19 lines · Frank Li
dw_pcie_ep_inbound_atu(){ ... if (!ep->bar_to_atu[bar]) free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows); else free_win = ep->bar_to_atu[bar]; ... }The atu index 0 is valid case for atu number. The
find_first_zero_bit()will return 6 when second time call into this function if atu is 0. Suppose it should use branch 'free_win= ep->bar_to_atu[bar]'.Change '
bar_to_atu' to s8. Initializebar_to_atuas -1 to indicate it have not allocate atu to the bar.Reported-by Niklas Cassel <[email protected]>Close: https://lore.kernel.org/linux-pci/ZXt2A+Fusfz3luQV@x1-carbon/T/#u
Quoted 8 lines
Fixes 4284c88fff0e ("PCI: designware-ep: Allow pci_epc_set_bar() update inbound map address")Signed-off-by Frank Li <[email protected]>---
Notes:
@Niklas:
I have not test your case. I should be equal to previous's fix in
mail list.Hello Frank,
Thank you for sending a proper fix for this!
Quoted 8 lines
Quoted 3 lines
Personally, I slightly prefer your fix that saves the iatu index + 1, and keeps 0 to mean unused. That way, you don't need the memset, and you don't need to change the type to signed, but either way is fine by me, so:
index + 1 don't match hardware iATU index. It will be confused because other parts is 0 based.
So I choose "-1" as free iATU.
A s8 can hold a max value of 127.CX_ATU_NUM_OUTBOUND_REGIONS seems to be 0-255.
Since the DWC code can be synthesized with 256 iATUs, your code will not work on systems with 128 or more iATUs.
If we continue to use a u8, and offset the saved value by one, we will at least be able to support 255-1 == 254 iATUs.
Kind regards,
Niklas
Hello Frank,
Quoted 44 lines · Niklas Cassel
Quoted 34 lines · Frank Li
Quoted 33 lines · Niklas Cassel
Quoted 19 lines · Frank Li
dw_pcie_ep_inbound_atu(){ ... if (!ep->bar_to_atu[bar]) free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows); else free_win = ep->bar_to_atu[bar]; ... }The atu index 0 is valid case for atu number. The
find_first_zero_bit()will return 6 when second time call into this function if atu is 0. Suppose it should use branch 'free_win= ep->bar_to_atu[bar]'.Change '
bar_to_atu' to s8. Initializebar_to_atuas -1 to indicate it have not allocate atu to the bar.Reported-by Niklas Cassel <[email protected]>Close: https://lore.kernel.org/linux-pci/ZXt2A+Fusfz3luQV@x1-carbon/T/#u
Quoted 8 lines
Fixes 4284c88fff0e ("PCI: designware-ep: Allow pci_epc_set_bar() update inbound map address")Signed-off-by Frank Li <[email protected]>---
Notes:
@Niklas:
I have not test your case. I should be equal to previous's fix in
mail list.Hello Frank,
Thank you for sending a proper fix for this!
Quoted 8 lines
Quoted 3 lines
Personally, I slightly prefer your fix that saves the iatu index + 1, and keeps 0 to mean unused. That way, you don't need the memset, and you don't need to change the type to signed, but either way is fine by me, so:
index + 1 don't match hardware iATU index. It will be confused because other parts is 0 based.
So I choose "-1" as free iATU.
Quoted 8 lines
A s8 can hold a max value of 127.
CX_ATU_NUM_OUTBOUND_REGIONSseems to be 0-255.Since the DWC code can be synthesized with 256 iATUs, your code will not work on systems with 128 or more iATUs.
If we continue to use a u8, and offset the saved value by one, we will at least be able to support 255-1 == 254 iATUs.
Do you plan to send out a v2?
Kind regards,
Niklas
Frank Li
maintainerQuoted 50 lines · Niklas Cassel
Hello Frank,
Quoted 47 lines · Niklas Cassel
Quoted 34 lines · Frank Li
Quoted 33 lines · Niklas Cassel
Quoted 19 lines · Frank Li
dw_pcie_ep_inbound_atu(){ ... if (!ep->bar_to_atu[bar]) free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows); else free_win = ep->bar_to_atu[bar]; ... }The atu index 0 is valid case for atu number. The
find_first_zero_bit()will return 6 when second time call into this function if atu is 0. Suppose it should use branch 'free_win= ep->bar_to_atu[bar]'.Change '
bar_to_atu' to s8. Initializebar_to_atuas -1 to indicate it have not allocate atu to the bar.Reported-by Niklas Cassel <[email protected]>Close: https://lore.kernel.org/linux-pci/ZXt2A+Fusfz3luQV@x1-carbon/T/#u
Quoted 8 lines
Fixes 4284c88fff0e ("PCI: designware-ep: Allow pci_epc_set_bar() update inbound map address")Signed-off-by Frank Li <[email protected]>---
Notes:
@Niklas:
I have not test your case. I should be equal to previous's fix in
mail list.Hello Frank,
Thank you for sending a proper fix for this!
Quoted 8 lines
Quoted 3 lines
Personally, I slightly prefer your fix that saves the iatu index + 1, and keeps 0 to mean unused. That way, you don't need the memset, and you don't need to change the type to signed, but either way is fine by me, so:
index + 1 don't match hardware iATU index. It will be confused because other parts is 0 based.
So I choose "-1" as free iATU.
A s8 can hold a max value of 127.
CX_ATU_NUM_OUTBOUND_REGIONSseems to be 0-255.
Quoted 7 lines
Quoted 5 lines
Since the DWC code can be synthesized with 256 iATUs, your code will not work on systems with 128 or more iATUs.
If we continue to use a u8, and offset the saved value by one, we will at least be able to support 255-1 == 254 iATUs.
Do you plan to send out a v2?
It is easy to change to u16. But I hope Manivannan Sadhasivam have time to review it.
Frank
Quoted 4 lines
Kind regards,
Niklas
Frank Li
maintainerQuoted 50 lines · Niklas Cassel
Hello Frank,
Quoted 47 lines · Niklas Cassel
Quoted 34 lines · Frank Li
Quoted 33 lines · Niklas Cassel
Quoted 19 lines · Frank Li
dw_pcie_ep_inbound_atu(){ ... if (!ep->bar_to_atu[bar]) free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows); else free_win = ep->bar_to_atu[bar]; ... }The atu index 0 is valid case for atu number. The
find_first_zero_bit()will return 6 when second time call into this function if atu is 0. Suppose it should use branch 'free_win= ep->bar_to_atu[bar]'.Change '
bar_to_atu' to s8. Initializebar_to_atuas -1 to indicate it have not allocate atu to the bar.Reported-by Niklas Cassel <[email protected]>Close: https://lore.kernel.org/linux-pci/ZXt2A+Fusfz3luQV@x1-carbon/T/#u
Quoted 8 lines
Fixes 4284c88fff0e ("PCI: designware-ep: Allow pci_epc_set_bar() update inbound map address")Signed-off-by Frank Li <[email protected]>---
Notes:
@Niklas:
I have not test your case. I should be equal to previous's fix in
mail list.Hello Frank,
Thank you for sending a proper fix for this!
Quoted 8 lines
Quoted 3 lines
Personally, I slightly prefer your fix that saves the iatu index + 1, and keeps 0 to mean unused. That way, you don't need the memset, and you don't need to change the type to signed, but either way is fine by me, so:
index + 1 don't match hardware iATU index. It will be confused because other parts is 0 based.
So I choose "-1" as free iATU.
A s8 can hold a max value of 127.
CX_ATU_NUM_OUTBOUND_REGIONSseems to be 0-255.
Quoted 7 lines
Quoted 5 lines
Since the DWC code can be synthesized with 256 iATUs, your code will not work on systems with 128 or more iATUs.
If we continue to use a u8, and offset the saved value by one, we will at least be able to support 255-1 == 254 iATUs.
Do you plan to send out a v2?
@mani:
Do you have any comments about this fixes except u8's problem?
Frank
Quoted 4 lines
Kind regards,
Niklas
Manivannan Sadhasivam
maintainerQuoted 54 lines · Frank Li
Quoted 53 lines · Niklas Cassel
Hello Frank,
Quoted 50 lines · Niklas Cassel
Quoted 34 lines · Frank Li
Quoted 33 lines · Niklas Cassel
Quoted 19 lines · Frank Li
dw_pcie_ep_inbound_atu(){ ... if (!ep->bar_to_atu[bar]) free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows); else free_win = ep->bar_to_atu[bar]; ... }The atu index 0 is valid case for atu number. The
find_first_zero_bit()will return 6 when second time call into this function if atu is 0. Suppose it should use branch 'free_win= ep->bar_to_atu[bar]'.Change '
bar_to_atu' to s8. Initializebar_to_atuas -1 to indicate it have not allocate atu to the bar.Reported-by Niklas Cassel <[email protected]>Close: https://lore.kernel.org/linux-pci/ZXt2A+Fusfz3luQV@x1-carbon/T/#u
Quoted 8 lines
Fixes 4284c88fff0e ("PCI: designware-ep: Allow pci_epc_set_bar() update inbound map address")Signed-off-by Frank Li <[email protected]>---
Notes:
@Niklas:
I have not test your case. I should be equal to previous's fix in
mail list.Hello Frank,
Thank you for sending a proper fix for this!
Quoted 8 lines
Quoted 3 lines
Personally, I slightly prefer your fix that saves the iatu index + 1, and keeps 0 to mean unused. That way, you don't need the memset, and you don't need to change the type to signed, but either way is fine by me, so:
index + 1 don't match hardware iATU index. It will be confused because other parts is 0 based.
So I choose "-1" as free iATU.
A s8 can hold a max value of 127.
CX_ATU_NUM_OUTBOUND_REGIONSseems to be 0-255.Since the DWC code can be synthesized with 256 iATUs, your code will not work on systems with 128 or more iATUs.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
If we continue to use a u8, and offset the saved value by one, we will at least be able to support 255-1 == 254 iATUs.
Do you plan to send out a v2?
@mani:
Do you have any comments about this fixes except u8's problem?
IMO, the core issue lies in the EPF driver. It calls set_bar() during init, but it is not clearing the BARs with clear_bar() during LINK_DOWN.
If the BARs were cleared properly, then we would not see this issue.
I'm planning to do a cleanup of the behavior of EPF with core_init_notifier and it should get addressed there.
- Mani
Signature · 1 line
மணிவண்ணன் சதாசிவம்
Quoted 63 lines · Manivannan Sadhasivam
Quoted 54 lines · Frank Li
Quoted 53 lines · Niklas Cassel
Hello Frank,
Quoted 50 lines · Niklas Cassel
Quoted 34 lines · Frank Li
Quoted 33 lines · Niklas Cassel
Quoted 19 lines · Frank Li
dw_pcie_ep_inbound_atu(){ ... if (!ep->bar_to_atu[bar]) free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows); else free_win = ep->bar_to_atu[bar]; ... }The atu index 0 is valid case for atu number. The
find_first_zero_bit()will return 6 when second time call into this function if atu is 0. Suppose it should use branch 'free_win= ep->bar_to_atu[bar]'.Change '
bar_to_atu' to s8. Initializebar_to_atuas -1 to indicate it have not allocate atu to the bar.Reported-by Niklas Cassel <[email protected]>Close: https://lore.kernel.org/linux-pci/ZXt2A+Fusfz3luQV@x1-carbon/T/#u
Quoted 8 lines
Fixes 4284c88fff0e ("PCI: designware-ep: Allow pci_epc_set_bar() update inbound map address")Signed-off-by Frank Li <[email protected]>---
Notes:
@Niklas:
I have not test your case. I should be equal to previous's fix in
mail list.Hello Frank,
Thank you for sending a proper fix for this!
Quoted 8 lines
Quoted 3 lines
Personally, I slightly prefer your fix that saves the iatu index + 1, and keeps 0 to mean unused. That way, you don't need the memset, and you don't need to change the type to signed, but either way is fine by me, so:
index + 1 don't match hardware iATU index. It will be confused because other parts is 0 based.
So I choose "-1" as free iATU.
A s8 can hold a max value of 127.
CX_ATU_NUM_OUTBOUND_REGIONSseems to be 0-255.Since the DWC code can be synthesized with 256 iATUs, your code will not work on systems with 128 or more iATUs.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
If we continue to use a u8, and offset the saved value by one, we will at least be able to support 255-1 == 254 iATUs.
Do you plan to send out a v2?
@mani:
Do you have any comments about this fixes except u8's problem?
Quoted 7 lines
IMO, the core issue lies in the EPF driver. It calls
set_bar()during init, but it is not clearing the BARs withclear_bar()duringLINK_DOWN.If the BARs were cleared properly, then we would not see this issue.
I'm planning to do a cleanup of the behavior of EPF with
core_init_notifierand it should get addressed there.
Hello Mani, Frank,
Please reconsider respinning this.
I think that the invalid usage of find_first_zero_bit() is very bad, because other people might copy the find_first_zero_bit() usage here.
find_first_zero_bit() can (and will) return 0, so this check is just bad:
if (!ep->bar_to_atu[bar])
free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows);
...
ep->bar_to_atu[bar] = free_win;set_bit(free_win, ep->ib_window_map);
No matter which code Mani adds that cleans up the BARs, will not change that the logic is this function is just broken.
Kind regards,
Niklas
Frank Li
maintainerQuoted 87 lines · Niklas Cassel
Quoted 63 lines · Manivannan Sadhasivam
Quoted 54 lines · Frank Li
Quoted 53 lines · Niklas Cassel
Hello Frank,
Quoted 50 lines · Niklas Cassel
Quoted 34 lines · Frank Li
Quoted 33 lines · Niklas Cassel
Quoted 19 lines · Frank Li
dw_pcie_ep_inbound_atu(){ ... if (!ep->bar_to_atu[bar]) free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows); else free_win = ep->bar_to_atu[bar]; ... }The atu index 0 is valid case for atu number. The
find_first_zero_bit()will return 6 when second time call into this function if atu is 0. Suppose it should use branch 'free_win= ep->bar_to_atu[bar]'.Change '
bar_to_atu' to s8. Initializebar_to_atuas -1 to indicate it have not allocate atu to the bar.Reported-by Niklas Cassel <[email protected]>Close: https://lore.kernel.org/linux-pci/ZXt2A+Fusfz3luQV@x1-carbon/T/#u
Quoted 8 lines
Fixes 4284c88fff0e ("PCI: designware-ep: Allow pci_epc_set_bar() update inbound map address")Signed-off-by Frank Li <[email protected]>---
Notes:
@Niklas:
I have not test your case. I should be equal to previous's fix in
mail list.Hello Frank,
Thank you for sending a proper fix for this!
Quoted 8 lines
Quoted 3 lines
Personally, I slightly prefer your fix that saves the iatu index + 1, and keeps 0 to mean unused. That way, you don't need the memset, and you don't need to change the type to signed, but either way is fine by me, so:
index + 1 don't match hardware iATU index. It will be confused because other parts is 0 based.
So I choose "-1" as free iATU.
A s8 can hold a max value of 127.
CX_ATU_NUM_OUTBOUND_REGIONSseems to be 0-255.Since the DWC code can be synthesized with 256 iATUs, your code will not work on systems with 128 or more iATUs.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
If we continue to use a u8, and offset the saved value by one, we will at least be able to support 255-1 == 254 iATUs.
Do you plan to send out a v2?
@mani:
Do you have any comments about this fixes except u8's problem?Quoted 7 lines
IMO, the core issue lies in the EPF driver. It calls
set_bar()during init, but it is not clearing the BARs withclear_bar()duringLINK_DOWN.If the BARs were cleared properly, then we would not see this issue.
I'm planning to do a cleanup of the behavior of EPF with
core_init_notifierand it should get addressed there.Hello Mani, Frank,
Please reconsider respinning this.
I think that the invalid usage of
find_first_zero_bit()is very bad, because other people might copy thefind_first_zero_bit()usage here.
find_first_zero_bit()can (and will) return 0, so this check is just bad:if (!ep->bar_to_atu[bar])
free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows);...
Quoted 7 lines
ep->bar_to_atu[bar] =
free_win;set_bit(free_win, ep->ib_window_map);No matter which code Mani adds that cleans up the BARs, will not change that the logic is this function is just broken.
Hello Mani:
if you agree this fix, I can respin this patch by fix s8 problem.
Frank
Quoted 4 lines
Kind regards,
Niklas
Manivannan Sadhasivam
maintainerQuoted 44 lines · Niklas Cassel
Quoted 34 lines · Frank Li
Quoted 33 lines · Niklas Cassel
Quoted 19 lines · Frank Li
dw_pcie_ep_inbound_atu(){ ... if (!ep->bar_to_atu[bar]) free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows); else free_win = ep->bar_to_atu[bar]; ... }The atu index 0 is valid case for atu number. The
find_first_zero_bit()will return 6 when second time call into this function if atu is 0. Suppose it should use branch 'free_win= ep->bar_to_atu[bar]'.Change '
bar_to_atu' to s8. Initializebar_to_atuas -1 to indicate it have not allocate atu to the bar.Reported-by Niklas Cassel <[email protected]>Close: https://lore.kernel.org/linux-pci/ZXt2A+Fusfz3luQV@x1-carbon/T/#u
Quoted 8 lines
Fixes 4284c88fff0e ("PCI: designware-ep: Allow pci_epc_set_bar() update inbound map address")Signed-off-by Frank Li <[email protected]>---
Notes:
@Niklas:
I have not test your case. I should be equal to previous's fix in
mail list.Hello Frank,
Thank you for sending a proper fix for this!
Quoted 8 lines
Quoted 3 lines
Personally, I slightly prefer your fix that saves the iatu index + 1, and keeps 0 to mean unused. That way, you don't need the memset, and you don't need to change the type to signed, but either way is fine by me, so:
index + 1 don't match hardware iATU index. It will be confused because other parts is 0 based.
So I choose "-1" as free iATU.
Quoted 9 lines
A s8 can hold a max value of 127.
CX_ATU_NUM_OUTBOUND_REGIONSseems to be 0-255.Since the DWC code can be synthesized with 256 iATUs, your code will not work on systems with 128 or more iATUs.
If we continue to use a u8, and offset the saved value by one, we will at least be able to support 255-1 == 254 iATUs.
Agree. I cannot suggest a better alternative. So let's go with this. But please add a comment before bar_to_atu assignment to make it clear. Like,
/* * Always increment free_win before assignment, since value 0 is used to * identify unallocated mapping. */ ep->bar_to_atu[bar] = free_win + 1;
- Mani
Quoted 3 lines
Kind regards,
Niklas
Signature · 1 line
மணிவண்ணன் சதாசிவம்
Frank Li
maintainerQuoted 55 lines · Manivannan Sadhasivam
Quoted 44 lines · Niklas Cassel
Quoted 34 lines · Frank Li
Quoted 33 lines · Niklas Cassel
Quoted 19 lines · Frank Li
dw_pcie_ep_inbound_atu(){ ... if (!ep->bar_to_atu[bar]) free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows); else free_win = ep->bar_to_atu[bar]; ... }The atu index 0 is valid case for atu number. The
find_first_zero_bit()will return 6 when second time call into this function if atu is 0. Suppose it should use branch 'free_win= ep->bar_to_atu[bar]'.Change '
bar_to_atu' to s8. Initializebar_to_atuas -1 to indicate it have not allocate atu to the bar.Reported-by Niklas Cassel <[email protected]>Close: https://lore.kernel.org/linux-pci/ZXt2A+Fusfz3luQV@x1-carbon/T/#u
Quoted 8 lines
Fixes 4284c88fff0e ("PCI: designware-ep: Allow pci_epc_set_bar() update inbound map address")Signed-off-by Frank Li <[email protected]>---
Notes:
@Niklas:
I have not test your case. I should be equal to previous's fix in
mail list.Hello Frank,
Thank you for sending a proper fix for this!
Quoted 8 lines
Quoted 3 lines
Personally, I slightly prefer your fix that saves the iatu index + 1, and keeps 0 to mean unused. That way, you don't need the memset, and you don't need to change the type to signed, but either way is fine by me, so:
index + 1 don't match hardware iATU index. It will be confused because other parts is 0 based.
So I choose "-1" as free iATU.
Quoted 9 lines
A s8 can hold a max value of 127.
CX_ATU_NUM_OUTBOUND_REGIONSseems to be 0-255.Since the DWC code can be synthesized with 256 iATUs, your code will not work on systems with 128 or more iATUs.
If we continue to use a u8, and offset the saved value by one, we will at least be able to support 255-1 == 254 iATUs.
Quoted 8 lines
Agree. I cannot suggest a better alternative. So let's go with this. But please add a comment before
bar_to_atuassignment to make it clear. Like,/* * Always increment free_win before assignment, since value 0 is used to * identify unallocated mapping. */ ep->bar_to_atu[bar] = free_win + 1;
This patch already change to use "-1" as free. Only issue for this patch is that use 's16' instead of 's8' becasue max ATU number is 255.
Frank
Quoted 9 lines
- Mani
Quoted 3 lines
Kind regards,
NiklasSignature · 1 line
மணிவண்ணன் சதாசிவம்
Manivannan Sadhasivam
maintainerQuoted 59 lines · Frank Li
Quoted 58 lines · Manivannan Sadhasivam
Quoted 44 lines · Niklas Cassel
Quoted 34 lines · Frank Li
Quoted 33 lines · Niklas Cassel
Quoted 19 lines · Frank Li
dw_pcie_ep_inbound_atu(){ ... if (!ep->bar_to_atu[bar]) free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows); else free_win = ep->bar_to_atu[bar]; ... }The atu index 0 is valid case for atu number. The
find_first_zero_bit()will return 6 when second time call into this function if atu is 0. Suppose it should use branch 'free_win= ep->bar_to_atu[bar]'.Change '
bar_to_atu' to s8. Initializebar_to_atuas -1 to indicate it have not allocate atu to the bar.Reported-by Niklas Cassel <[email protected]>Close: https://lore.kernel.org/linux-pci/ZXt2A+Fusfz3luQV@x1-carbon/T/#u
Quoted 8 lines
Fixes 4284c88fff0e ("PCI: designware-ep: Allow pci_epc_set_bar() update inbound map address")Signed-off-by Frank Li <[email protected]>---
Notes:
@Niklas:
I have not test your case. I should be equal to previous's fix in
mail list.Hello Frank,
Thank you for sending a proper fix for this!
Quoted 8 lines
Quoted 3 lines
Personally, I slightly prefer your fix that saves the iatu index + 1, and keeps 0 to mean unused. That way, you don't need the memset, and you don't need to change the type to signed, but either way is fine by me, so:
index + 1 don't match hardware iATU index. It will be confused because other parts is 0 based.
So I choose "-1" as free iATU.
Quoted 9 lines
A s8 can hold a max value of 127.
CX_ATU_NUM_OUTBOUND_REGIONSseems to be 0-255.Since the DWC code can be synthesized with 256 iATUs, your code will not work on systems with 128 or more iATUs.
If we continue to use a u8, and offset the saved value by one, we will at least be able to support 255-1 == 254 iATUs.
Agree. I cannot suggest a better alternative. So let's go with this. But please add a comment before
bar_to_atuassignment to make it clear. Like,
Quoted 9 lines
Quoted 5 lines
/* * Always increment free_win before assignment, since value 0 is used to * identify unallocated mapping. */ ep->bar_to_atu[bar] = free_win + 1;This patch already change to use "-1" as free. Only issue for this patch is that use 's16' instead of 's8' becasue max ATU number is 255.
Niklas's initial suggestion of keeping u8 for the array and 0 as the unallocated placeholder sounds good to me. Please use that instead.
Even though iATU window index starts from 0, the comment I suggested will clarify what this bar_to_atu[] does.
- Mani
Signature · 1 line
மணிவண்ணன் சதாசிவம்
Frank Li
maintainerQuoted 66 lines · Manivannan Sadhasivam
Quoted 65 lines · Frank Li
Quoted 63 lines · Manivannan Sadhasivam
Quoted 44 lines · Niklas Cassel
Quoted 34 lines · Frank Li
Quoted 33 lines · Niklas Cassel
Quoted 19 lines · Frank Li
dw_pcie_ep_inbound_atu(){ ... if (!ep->bar_to_atu[bar]) free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows); else free_win = ep->bar_to_atu[bar]; ... }The atu index 0 is valid case for atu number. The
find_first_zero_bit()will return 6 when second time call into this function if atu is 0. Suppose it should use branch 'free_win= ep->bar_to_atu[bar]'.Change '
bar_to_atu' to s8. Initializebar_to_atuas -1 to indicate it have not allocate atu to the bar.Reported-by Niklas Cassel <[email protected]>Close: https://lore.kernel.org/linux-pci/ZXt2A+Fusfz3luQV@x1-carbon/T/#u
Quoted 8 lines
Fixes 4284c88fff0e ("PCI: designware-ep: Allow pci_epc_set_bar() update inbound map address")Signed-off-by Frank Li <[email protected]>---
Notes:
@Niklas:
I have not test your case. I should be equal to previous's fix in
mail list.Hello Frank,
Thank you for sending a proper fix for this!
Quoted 8 lines
Quoted 3 lines
Personally, I slightly prefer your fix that saves the iatu index + 1, and keeps 0 to mean unused. That way, you don't need the memset, and you don't need to change the type to signed, but either way is fine by me, so:
index + 1 don't match hardware iATU index. It will be confused because other parts is 0 based.
So I choose "-1" as free iATU.
Quoted 9 lines
A s8 can hold a max value of 127.
CX_ATU_NUM_OUTBOUND_REGIONSseems to be 0-255.Since the DWC code can be synthesized with 256 iATUs, your code will not work on systems with 128 or more iATUs.
If we continue to use a u8, and offset the saved value by one, we will at least be able to support 255-1 == 254 iATUs.
Agree. I cannot suggest a better alternative. So let's go with this. But please add a comment before
bar_to_atuassignment to make it clear. Like,/* * Always increment free_win before assignment, since value 0 is used to * identify unallocated mapping. */ ep->bar_to_atu[bar] = free_win + 1;
Quoted 7 lines
Quoted 3 lines
This patch already change to use "-1" as free. Only issue for this patch is that use 's16' instead of 's8' becasue max ATU number is 255.
Niklas's initial suggestion of keeping u8 for the array and 0 as the unallocated placeholder sounds good to me. Please use that instead.
It is impossible to keep u8, because 255 + 1 will 0 for u8. Previously Niklas's initial suggestion have not consider this condition. If u8 have to change to u16 or s16 anyways, I prefer use -1 as free.
Frank
Quoted 7 lines
Even though iATU window index starts from 0, the comment I suggested will clarify what this bar_to_atu[] does.
- Mani
Signature · 1 line
மணிவண்ணன் சதாசிவம்
Quoted 8 lines · Frank Li
Quoted 4 lines
Niklas's initial suggestion of keeping u8 for the array and 0 as the unallocated placeholder sounds good to me. Please use that instead.
It is impossible to keep u8, because 255 + 1 will 0 for u8. Previously Niklas's initial suggestion have not consider this condition. If u8 have to change to u16 or s16 anyways, I prefer use -1 as free.
Well, to be fair, my suggestion was:
"If we continue to use a u8, and offset the saved value by one, we will at least be able to support 255-1 == 254 iATUs."
But we have this define:
drivers/pci/controller/dwc/pcie-designware.h:#define MAX_IATU_IN 256 (Even if it isn't used anywhere.)
But as ridiculous as it may seem to support that many inbound ranges, that is the max number of windows supported by the hardware, so why not just let the driver support the max supported by the hardware?
We are talking about:
struct dw_pcie_ep {
...
u8 bar_to_atu[PCI_STD_NUM_BARS];
...
}where PCI_STD_NUM_BARS == 6.
And where struct dw_pcie_ep is kzalloced for what I assume is all drivers.
So I'm actually for your idea of changing it to u16, or even unsigned int.
If the code is simplified if we use a u16 or unsigned int (because we don't need any weird if (val < MAX_IATU_IN - 1) check), then I'm all for it.
What I personally don't like with your patch in $subject,
was that you changed both dw_pcie_ep_clear_bar() to set the "clear value" to -1, but you also need amemset(ep->bar_to_atu, -1, sizeof(ep->bar_to_atu)); in dw_pcie_ep_init().
I much prefer to have 0 as the default/unused value, because it will automatically get set when you do kzalloc().
Seeing a memset -1 just looks a bit hackish to be, but I realize that it is personal preference.
If it is super important to save 8 bytes from the heap, then I would even prefer changing the MAX_IATU_IN 256 to something smaller, like 127 or whatever, just as long as we don't need that extra memset -1 :)
Kind regards,
Niklas
Frank Li
maintainerQuoted 52 lines · Niklas Cassel
Quoted 8 lines · Frank Li
Quoted 4 lines
Niklas's initial suggestion of keeping u8 for the array and 0 as the unallocated placeholder sounds good to me. Please use that instead.
It is impossible to keep u8, because 255 + 1 will 0 for u8. Previously Niklas's initial suggestion have not consider this condition. If u8 have to change to u16 or s16 anyways, I prefer use -1 as free.
Well, to be fair, my suggestion was:
"If we continue to use a u8, and offset the saved value by one, we will at least be able to support 255-1 == 254 iATUs."But we have this define:
drivers/pci/controller/dwc/pcie-designware.h:#defineMAX_IATU_IN256 (Even if it isn't used anywhere.)But as ridiculous as it may seem to support that many inbound ranges, that is the max number of windows supported by the hardware, so why not just let the driver support the max supported by the hardware?
We are talking about:
struct dw_pcie_ep { ... u8 bar_to_atu[PCI_STD_NUM_BARS]; ... }where
PCI_STD_NUM_BARS== 6.And where struct
dw_pcie_epis kzalloced for what I assume is all drivers.So I'm actually for your idea of changing it to u16, or even unsigned int.
If the code is simplified if we use a u16 or unsigned int (because we don't need any weird if (val <
MAX_IATU_IN- 1) check), then I'm all for it.What I personally don't like with your patch in $subject,
was that you changed bothdw_pcie_ep_clear_bar()to set the "clear value" to -1, but you also need amemset(ep->bar_to_atu, -1, sizeof(ep->bar_to_atu)); indw_pcie_ep_init().I much prefer to have 0 as the default/unused value, because it will automatically get set when you do
kzalloc().
Seeing a memset -1 just looks a bit hackish to be, but I realize that it is personal preference.
Quoted 3 lines
If it is super important to save 8 bytes from the heap, then I would even prefer changing the
MAX_IATU_IN256 to something smaller, like 127 or whatever, just as long as we don't need that extra memset -1 :)
Okay, Let me work on '0' version.
Frank
Quoted 4 lines
Kind regards,
Niklas