Skip to contentSkip to conversation

Add PAX_REFCOUNT overflow protection

RFC PATCH Superseded David Windsor 2 participants 6 messages Dec 17, 2015

Subsystems (12)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Add PAX_REFCOUNT overflow protection

These patches are in support of the kernel self-protection effort recently proposed on kernel-hardening.

The Linux kernel can be better protected against use-after-free exploits. This patch adds functionality originally found in the PaX project [1] to protect the kernel against integer overflows affecting kernel reference counters, a class of bugs that can lead to use-after-free exploits.

Problems arise when reference counters get overflowed. The existence of a kernel path in which a reference counter is incremented more than it is decremented leads to an overflow bug. This buggy path can be executed until INT_MAX is reached, at which point further increments will cause the reference counter to wrap around to 0. At this point, the kernel will free the counted object while it still has legitimate users, creating conditions for a use-after-free exploit to occur.

Patch Details
----------------

These patches were taken from the PaX project's PAX_REFCOUNT feature. It applies cleanly to linux-stable/v4.2.6 [2]. I am in the process of rebasing these patches to linux-next, but wanted to submit them now for general comments, bikeshedding, etc.

Most of the changes involve users of atomic_t opting out of overflow protection:
many atomic_t users use the type not as a reference counter, but as some type of statistical counter (packet counters, etc.).

Issues
---------

* The patch is currently based on linux-stable/v4.2.6. I am in the process of rebasing it to linux-next and will submit a follow-up patchset soon.

* PAX_REFCOUNT naming: I picked this naming scheme by default. It will likely need to change before going upstream.

* When an overflow is detected, SIGKILL is sent to the offending process. This may be too drastic for an initial upstream submission. WARN_ON may be more appropriate until distros have some time to absorb it and report any unaddressed overflows.

* 0003-pax-opt-out-of-PAX_REFCOUNT-protection.patch is very large. I have another patchset in which these patches are split more or less according to maintainer (except for drivers/) [3].

[1] https://forums.grsecurity.net/viewtopic.php?f=7&t=4173
[2] git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
[3] https://github.com/dwindsor/linux-stable/tree/dwindsor-pax-refcount-split

David Windsor (5):
x86: add PAX_REFCOUNT support
percpu_ref: decrease per-CPU refcount bias
pax: opt out of PAX_REFCOUNT protection
security: add Kconfig options for PAX_REFCOUNT
lkdtm: add test for atomic_t underflow/overflow

 arch/x86/include/asm/atomic.h             | 263 ++++++++++++++++++++--
 arch/x86/include/asm/atomic64_32.h        | 100 +++++++++
 arch/x86/include/asm/atomic64_64.h        | 164 +++++++++++++-
 arch/x86/include/asm/bitops.h             |   6 +-
 arch/x86/include/asm/cmpxchg.h            |  39 ++++
 arch/x86/include/asm/hw_irq.h             |   4 +-
 arch/x86/include/asm/local.h              | 106 ++++++++-
 arch/x86/include/asm/preempt.h            |   2 +-
 arch/x86/include/asm/rmwcc.h              |  84 +++++++-
 arch/x86/include/asm/rwsem.h              |  60 +++++-
 arch/x86/kernel/apic/apic.c               |   2 +-
 arch/x86/kernel/apic/io_apic.c            |   4 +-
 arch/x86/kernel/cpu/mcheck/mce.c          |  12 +-
 arch/x86/kernel/i8259.c                   |   2 +-
 arch/x86/kernel/irq.c                     |   8 +-
 arch/x86/kernel/kgdb.c                    |   6 +-
 arch/x86/kernel/pvclock.c                 |   8 +-
 arch/x86/kernel/tboot.c                   |   8 +-
 arch/x86/kernel/traps.c                   |   6 +
 arch/x86/lib/atomic64_386_32.S            | 164 ++++++++++++++
 arch/x86/lib/atomic64_cx8_32.S            |  94 +++++++-
 arch/x86/mm/mmio-mod.c                    |   4 +-
 drivers/acpi/apei/ghes.c                  |   4 +-
 drivers/ata/libata-core.c                 |   4 +-
 drivers/ata/libata-scsi.c                 |   2 +-
 drivers/ata/libata.h                      |   2 +-
 drivers/atm/adummy.c                      |   2 +-
 drivers/atm/ambassador.c                  |   8 +-
 drivers/atm/atmtcp.c                      |  14 +-
 drivers/atm/eni.c                         |  10 +-
 drivers/atm/firestream.c                  |   8 +-
 drivers/atm/fore200e.c                    |  14 +-
 drivers/atm/he.c                          |  18 +-
 drivers/atm/horizon.c                     |   4 +-
 drivers/atm/idt77252.c                    |  36 ++--
 drivers/atm/iphase.c                      |  34 +--
 drivers/atm/lanai.c                       |  12 +-
 drivers/atm/nicstar.c                     |  46 ++--
 drivers/atm/solos-pci.c                   |   4 +-
 drivers/atm/suni.c                        |   4 +-
 drivers/atm/uPD98402.c                    |  16 +-
 drivers/atm/zatm.c                        |   6 +-
 drivers/base/power/wakeup.c               |   8 +-
 drivers/block/drbd/drbd_bitmap.c          |   2 +-
 drivers/block/drbd/drbd_int.h             |   8 +-
 drivers/block/drbd/drbd_main.c            |  12 +-
 drivers/block/drbd/drbd_nl.c              |   4 +-
 drivers/block/drbd/drbd_receiver.c        |  30 +--
 drivers/block/drbd/drbd_worker.c          |   8 +-
 drivers/char/ipmi/ipmi_msghandler.c       |   8 +-
 drivers/char/ipmi/ipmi_si_intf.c          |   8 +-
 drivers/crypto/hifn_795x.c                |   4 +-
 drivers/edac/edac_device.c                |   4 +-
 drivers/edac/edac_pci.c                   |   4 +-
 drivers/edac/edac_pci_sysfs.c             |  20 +-
 drivers/firewire/core-card.c              |   4 +-
 drivers/firmware/efi/cper.c               |   8 +-
 drivers/gpio/gpio-vr41xx.c                |   2 +-
 drivers/gpu/drm/i810/i810_drv.h           |   4 +-
 drivers/gpu/drm/mga/mga_drv.h             |   4 +-
 drivers/gpu/drm/mga/mga_irq.c             |   8 +-
 drivers/gpu/drm/qxl/qxl_cmd.c             |  12 +-
 drivers/gpu/drm/qxl/qxl_debugfs.c         |   8 +-
 drivers/gpu/drm/qxl/qxl_drv.h             |   8 +-
 drivers/gpu/drm/qxl/qxl_irq.c             |  16 +-
 drivers/gpu/drm/r128/r128_cce.c           |   2 +-
 drivers/gpu/drm/r128/r128_drv.h           |   4 +-
 drivers/gpu/drm/r128/r128_irq.c           |   4 +-
 drivers/gpu/drm/r128/r128_state.c         |   4 +-
 drivers/gpu/drm/radeon/radeon_drv.h       |   2 +-
 drivers/gpu/drm/radeon/radeon_irq.c       |   6 +-
 drivers/gpu/drm/via/via_drv.h             |   4 +-
 drivers/gpu/drm/via/via_irq.c             |  18 +-
 drivers/gpu/drm/vmwgfx/vmwgfx_drv.h       |   2 +-
 drivers/gpu/drm/vmwgfx/vmwgfx_fifo.c      |   6 +-
 drivers/gpu/drm/vmwgfx/vmwgfx_irq.c       |   4 +-
 drivers/gpu/drm/vmwgfx/vmwgfx_marker.c    |   2 +-
 drivers/hid/hid-core.c                    |   4 +-
 drivers/hv/channel.c                      |   2 +-
 drivers/hv/hv_balloon.c                   |  18 +-
 drivers/hv/hyperv_vmbus.h                 |   2 +-
 drivers/hwmon/sht15.c                     |  12 +-
 drivers/infiniband/core/cm.c              |  32 +--
 drivers/infiniband/core/fmr_pool.c        |  20 +-
 drivers/infiniband/hw/cxgb4/mem.c         |   4 +-
 drivers/infiniband/hw/ipath/ipath_rc.c    |   6 +-
 drivers/infiniband/hw/ipath/ipath_ruc.c   |   6 +-
 drivers/infiniband/hw/mlx4/mad.c          |   2 +-
 drivers/infiniband/hw/mlx4/mcg.c          |   2 +-
 drivers/infiniband/hw/mlx4/mlx4_ib.h      |   2 +-
 drivers/infiniband/hw/nes/nes.c           |   4 +-
 drivers/infiniband/hw/nes/nes.h           |  40 ++--
 drivers/infiniband/hw/nes/nes_cm.c        |  62 +++---
 drivers/infiniband/hw/nes/nes_mgt.c       |   8 +-
 drivers/infiniband/hw/nes/nes_nic.c       |  40 ++--
 drivers/infiniband/hw/nes/nes_verbs.c     |  10 +-
 drivers/input/gameport/gameport.c         |   4 +-
 drivers/input/input.c                     |   4 +-
 drivers/input/joystick/xpad.c             |   4 +-
 drivers/input/misc/ims-pcu.c              |   4 +-
 drivers/input/serio/serio.c               |   4 +-
 drivers/input/serio/serio_raw.c           |   4 +-
 drivers/isdn/capi/capi.c                  |  10 +-
 drivers/md/dm-raid1.c                     |  18 +-
 drivers/md/dm-stripe.c                    |  10 +-
 drivers/md/dm.c                           |  16 +-
 drivers/md/md.c                           |  26 +--
 drivers/md/md.h                           |   6 +-
 drivers/md/raid1.c                        |   4 +-
 drivers/md/raid10.c                       |  16 +-
 drivers/md/raid5.c                        |  10 +-
 drivers/media/pci/ivtv/ivtv-driver.c      |   2 +-
 drivers/media/pci/solo6x10/solo6x10-p2m.c |   2 +-
 drivers/media/pci/solo6x10/solo6x10.h     |   2 +-
 drivers/media/pci/tw68/tw68-core.c        |   2 +-
 drivers/media/radio/radio-maxiradio.c     |   2 +-
 drivers/media/radio/radio-shark.c         |   2 +-
 drivers/media/radio/radio-shark2.c        |   2 +-
 drivers/media/radio/radio-si476x.c        |   2 +-
 drivers/media/v4l2-core/v4l2-device.c     |   4 +-
 drivers/misc/lis3lv02d/lis3lv02d.c        |   8 +-
 drivers/misc/lis3lv02d/lis3lv02d.h        |   2 +-
 drivers/misc/lkdtm.c                      |  13 ++
 drivers/misc/sgi-gru/gruhandles.c         |   4 +-
 drivers/misc/sgi-gru/gruprocfs.c          |   8 +-
 drivers/misc/sgi-gru/grutables.h          | 158 +++++++-------
 drivers/net/hyperv/hyperv_net.h           |   2 +-
 drivers/net/hyperv/rndis_filter.c         |   4 +-
 drivers/net/ipvlan/ipvlan_core.c          |   2 +-
 drivers/net/macvlan.c                     |   2 +-
 drivers/net/usb/sierra_net.c              |   4 +-
 drivers/net/wireless/rt2x00/rt2x00.h      |   2 +-
 drivers/net/wireless/rt2x00/rt2x00queue.c |   4 +-
 drivers/oprofile/buffer_sync.c            |   8 +-
 drivers/oprofile/event_buffer.c           |   2 +-
 drivers/oprofile/oprof.c                  |   2 +-
 drivers/oprofile/oprofile_stats.c         |  10 +-
 drivers/oprofile/oprofile_stats.h         |  10 +-
 drivers/oprofile/oprofilefs.c             |   6 +-
 drivers/regulator/core.c                  |   4 +-
 drivers/scsi/fcoe/fcoe_sysfs.c            |  12 +-
 drivers/scsi/hosts.c                      |   4 +-
 drivers/scsi/libfc/fc_exch.c              |  50 ++---
 drivers/scsi/lpfc/lpfc.h                  |   8 +-
 drivers/scsi/lpfc/lpfc_debugfs.c          |  18 +-
 drivers/scsi/lpfc/lpfc_scsi.c             |  10 +-
 drivers/scsi/pmcraid.c                    |  20 +-
 drivers/scsi/pmcraid.h                    |   8 +-
 drivers/scsi/qla4xxx/ql4_def.h            |   2 +-
 drivers/scsi/qla4xxx/ql4_os.c             |   6 +-
 drivers/scsi/scsi_lib.c                   |   8 +-
 drivers/scsi/scsi_sysfs.c                 |   2 +-
 drivers/scsi/scsi_transport_fc.c          |   6 +-
 drivers/scsi/scsi_transport_iscsi.c       |   6 +-
 drivers/scsi/scsi_transport_srp.c         |   6 +-
 drivers/scsi/sd.c                         |   2 +-
 drivers/staging/android/timed_output.c    |   6 +-
 drivers/staging/octeon/ethernet-rx.c      |  12 +-
 drivers/staging/octeon/ethernet.c         |   8 +-
 drivers/target/sbp/sbp_target.c           |   4 +-
 drivers/target/target_core_device.c       |   2 +-
 drivers/target/target_core_transport.c    |   2 +-
 drivers/tty/hvc/hvsi.c                    |  12 +-
 drivers/tty/hvc/hvsi_lib.c                |   4 +-
 drivers/tty/serial/ioc4_serial.c          |   6 +-
 drivers/tty/serial/msm_serial.c           |   4 +-
 drivers/uio/uio.c                         |  12 +-
 drivers/usb/atm/usbatm.c                  |  24 +--
 drivers/usb/core/devices.c                |   6 +-
 drivers/usb/core/hcd.c                    |   4 +-
 drivers/usb/core/sysfs.c                  |   2 +-
 drivers/usb/core/usb.c                    |   2 +-
 drivers/usb/host/ehci-hub.c               |   4 +-
 drivers/usb/misc/appledisplay.c           |   4 +-
 drivers/usb/usbip/vhci.h                  |   2 +-
 drivers/usb/usbip/vhci_hcd.c              |   6 +-
 drivers/usb/usbip/vhci_rx.c               |   2 +-
 drivers/usb/wusbcore/wa-hc.h              |   4 +-
 drivers/usb/wusbcore/wa-xfer.c            |   2 +-
 drivers/video/fbdev/hyperv_fb.c           |   4 +-
 drivers/video/fbdev/udlfb.c               |  32 +--
 fs/afs/inode.c                            |   4 +-
 fs/btrfs/delayed-inode.c                  |   6 +-
 fs/btrfs/delayed-inode.h                  |   4 +-
 fs/cachefiles/internal.h                  |  12 +-
 fs/cachefiles/namei.c                     |   2 +-
 fs/cachefiles/proc.c                      |  12 +-
 fs/ceph/super.c                           |   4 +-
 fs/cifs/cifs_debug.c                      |  12 +-
 fs/cifs/cifsfs.c                          |   4 +-
 fs/cifs/cifsglob.h                        |  54 ++---
 fs/cifs/misc.c                            |   4 +-
 fs/cifs/smb1ops.c                         |  80 +++----
 fs/cifs/smb2ops.c                         |  84 ++++----
 fs/coda/cache.c                           |  10 +-
 fs/coredump.c                             |   6 +-
 fs/exec.c                                 |  23 ++
 fs/ext4/ext4.h                            |  20 +-
 fs/ext4/mballoc.c                         |  44 ++--
 fs/fscache/cookie.c                       |  40 ++--
 fs/fscache/internal.h                     | 202 ++++++++---------
 fs/fscache/object.c                       |  26 +--
 fs/fscache/operation.c                    |  38 ++--
 fs/fscache/page.c                         | 110 +++++-----
 fs/fscache/stats.c                        | 348 +++++++++++++++---------------
 fs/inode.c                                |   4 +-
 fs/kernfs/file.c                          |  12 +-
 fs/lockd/clntproc.c                       |   4 +-
 fs/namespace.c                            |   4 +-
 fs/nfs/inode.c                            |   6 +-
 fs/notify/notification.c                  |   4 +-
 fs/ocfs2/localalloc.c                     |   2 +-
 fs/ocfs2/ocfs2.h                          |  10 +-
 fs/ocfs2/suballoc.c                       |  12 +-
 fs/ocfs2/super.c                          |  20 +-
 fs/proc/meminfo.c                         |   2 +-
 fs/quota/netlink.c                        |   4 +-
 fs/reiserfs/do_balan.c                    |   2 +-
 fs/reiserfs/procfs.c                      |   2 +-
 fs/reiserfs/reiserfs.h                    |   4 +-
 include/asm-generic/atomic-long.h         | 210 ++++++++++++++++++
 include/asm-generic/atomic64.h            |  12 ++
 include/asm-generic/local.h               |  13 ++
 include/linux/atmdev.h                    |   2 +-
 include/linux/blktrace_api.h              |   2 +-
 include/linux/fscache-cache.h             |   2 +-
 include/linux/genhd.h                     |   2 +-
 include/linux/irqdesc.h                   |   2 +-
 include/linux/kgdb.h                      |   2 +-
 include/linux/netdevice.h                 |   6 +-
 include/linux/oprofile.h                  |   4 +-
 include/linux/padata.h                    |   2 +-
 include/linux/perf_event.h                |   8 +-
 include/linux/sched.h                     |  15 +-
 include/linux/sonet.h                     |   2 +-
 include/linux/sunrpc/svc_rdma.h           |  18 +-
 include/linux/types.h                     |  16 ++
 include/linux/uio_driver.h                |   2 +-
 include/linux/usb.h                       |   2 +-
 include/media/v4l2-device.h               |   2 +-
 include/net/bonding.h                     |   2 +-
 include/net/caif/cfctrl.h                 |   4 +-
 include/net/flow.h                        |   2 +-
 include/net/gro_cells.h                   |   2 +-
 include/net/inetpeer.h                    |   2 +-
 include/net/ip_fib.h                      |   2 +-
 include/net/ip_vs.h                       |   4 +-
 include/net/iucv/af_iucv.h                |   2 +-
 include/net/net_namespace.h               |  12 +-
 include/net/netns/ipv4.h                  |   4 +-
 include/net/netns/ipv6.h                  |   4 +-
 include/net/netns/xfrm.h                  |   2 +-
 include/net/sock.h                        |   6 +-
 include/net/xfrm.h                        |   2 +-
 include/scsi/scsi_device.h                |   6 +-
 include/target/target_core_base.h         |   2 +-
 include/video/udlfb.h                     |   8 +-
 kernel/audit.c                            |   8 +-
 kernel/auditsc.c                          |   4 +-
 kernel/debug/debug_core.c                 |  16 +-
 kernel/events/core.c                      |  22 +-
 kernel/irq/manage.c                       |   2 +-
 kernel/irq/spurious.c                     |   2 +-
 kernel/locking/lockdep.c                  |   2 +-
 kernel/locking/rtmutex-tester.c           |  24 +--
 kernel/padata.c                           |   4 +-
 kernel/profile.c                          |  14 +-
 kernel/rcu/rcutorture.c                   |  60 +++---
 kernel/rcu/tree.c                         |  64 +++---
 kernel/rcu/tree.h                         |  26 +--
 kernel/rcu/tree_plugin.h                  |  12 +-
 kernel/rcu/tree_trace.c                   |  22 +-
 kernel/sched/auto_group.c                 |   4 +-
 kernel/time/timer_stats.c                 |  10 +-
 kernel/trace/blktrace.c                   |   6 +-
 kernel/trace/ftrace.c                     |   4 +-
 kernel/trace/ring_buffer.c                |  96 ++++-----
 kernel/trace/trace_clock.c                |   4 +-
 kernel/trace/trace_functions_graph.c      |   4 +-
 kernel/trace/trace_mmiotrace.c            |   8 +-
 lib/percpu-refcount.c                     |   2 +-
 lib/show_mem.c                            |   2 +-
 net/atm/atm_misc.c                        |   8 +-
 net/atm/proc.c                            |   6 +-
 net/atm/resources.c                       |   4 +-
 net/batman-adv/bat_iv_ogm.c               |   8 +-
 net/batman-adv/fragmentation.c            |   2 +-
 net/batman-adv/soft-interface.c           |   6 +-
 net/batman-adv/types.h                    |   6 +-
 net/caif/cfctrl.c                         |  10 +-
 net/ceph/messenger.c                      |   4 +-
 net/core/datagram.c                       |   2 +-
 net/core/dev.c                            |  12 +-
 net/core/flow.c                           |   6 +-
 net/core/net-sysfs.c                      |   2 +-
 net/core/netpoll.c                        |   4 +-
 net/core/rtnetlink.c                      |   2 +-
 net/core/sock.c                           |  12 +-
 net/core/sock_diag.c                      |   6 +-
 net/ipv4/devinet.c                        |   4 +-
 net/ipv4/fib_frontend.c                   |   6 +-
 net/ipv4/fib_semantics.c                  |   2 +-
 net/ipv4/inet_connection_sock.c           |   4 +-
 net/ipv4/inet_timewait_sock.c             |   2 +-
 net/ipv4/inetpeer.c                       |   2 +-
 net/ipv4/ip_fragment.c                    |   2 +-
 net/ipv4/ping.c                           |   2 +-
 net/ipv4/raw.c                            |   4 +-
 net/ipv4/route.c                          |  10 +-
 net/ipv4/tcp_input.c                      |   2 +-
 net/ipv4/udp.c                            |  10 +-
 net/ipv6/addrconf.c                       |   6 +-
 net/ipv6/af_inet6.c                       |   2 +-
 net/ipv6/datagram.c                       |   2 +-
 net/ipv6/ip6_fib.c                        |   4 +-
 net/ipv6/raw.c                            |   6 +-
 net/ipv6/udp.c                            |   6 +-
 net/iucv/af_iucv.c                        |   4 +-
 net/key/af_key.c                          |   4 +-
 net/l2tp/l2tp_eth.c                       |  38 ++--
 net/netfilter/ipvs/ip_vs_conn.c           |   6 +-
 net/netfilter/ipvs/ip_vs_core.c           |   4 +-
 net/netfilter/ipvs/ip_vs_ctl.c            |  10 +-
 net/netfilter/ipvs/ip_vs_sync.c           |   6 +-
 net/netfilter/ipvs/ip_vs_xmit.c           |   4 +-
 net/netfilter/nfnetlink_log.c             |   4 +-
 net/netfilter/xt_statistic.c              |   8 +-
 net/netlink/af_netlink.c                  |   4 +-
 net/openvswitch/vport.c                   |  16 +-
 net/openvswitch/vport.h                   |   8 +-
 net/packet/af_packet.c                    |   4 +-
 net/phonet/pep.c                          |   6 +-
 net/phonet/socket.c                       |   2 +-
 net/rds/cong.c                            |   6 +-
 net/rds/ib.h                              |   2 +-
 net/rds/ib_cm.c                           |   2 +-
 net/rds/ib_recv.c                         |   4 +-
 net/rds/iw.h                              |   2 +-
 net/rds/iw_cm.c                           |   2 +-
 net/rds/iw_recv.c                         |   4 +-
 net/rxrpc/af_rxrpc.c                      |   2 +-
 net/rxrpc/ar-ack.c                        |  14 +-
 net/rxrpc/ar-call.c                       |   2 +-
 net/rxrpc/ar-connection.c                 |   2 +-
 net/rxrpc/ar-connevent.c                  |   2 +-
 net/rxrpc/ar-input.c                      |   4 +-
 net/rxrpc/ar-internal.h                   |   8 +-
 net/rxrpc/ar-local.c                      |   2 +-
 net/rxrpc/ar-output.c                     |   4 +-
 net/rxrpc/ar-peer.c                       |   2 +-
 net/rxrpc/ar-proc.c                       |   4 +-
 net/rxrpc/ar-transport.c                  |   2 +-
 net/rxrpc/rxkad.c                         |   4 +-
 net/sched/sch_generic.c                   |   4 +-
 net/sunrpc/auth_gss/svcauth_gss.c         |   4 +-
 net/sunrpc/sched.c                        |   4 +-
 net/sunrpc/xprtrdma/svc_rdma.c            |  36 ++--
 net/sunrpc/xprtrdma/svc_rdma_recvfrom.c   |   8 +-
 net/sunrpc/xprtrdma/svc_rdma_sendto.c     |   2 +-
 net/sunrpc/xprtrdma/svc_rdma_transport.c  |  10 +-
 net/xfrm/xfrm_policy.c                    |  10 +-
 net/xfrm/xfrm_state.c                     |   4 +-
 security/Kconfig                          |  72 +++++++
 security/integrity/ima/ima.h              |   4 +-
 security/integrity/ima/ima_api.c          |   2 +-
 security/integrity/ima/ima_fs.c           |   4 +-
 security/integrity/ima/ima_queue.c        |   2 +-
 security/selinux/avc.c                    |   6 +-
 security/selinux/include/xfrm.h           |   2 +-
 sound/pci/ymfpci/ymfpci.h                 |   2 +-
 sound/pci/ymfpci/ymfpci_main.c            |  12 +-
 virt/kvm/kvm_main.c                       |   8 +-
 372 files changed, 3325 insertions(+), 1977 deletions(-)
Signature · 1 line
2.5.0
#1 lore raw

RFC PATCH 1/5 x86: add PAX_REFCOUNT support

Add overflow protection to atomic_t and create a new type atomic_unchecked_t for users of atomic_t not using the type for reference counting.

---
 arch/x86/include/asm/atomic.h      | 263 ++++++++++++++++++++++++++++++++++---
 arch/x86/include/asm/atomic64_32.h | 100 ++++++++++++++
 arch/x86/include/asm/atomic64_64.h | 164 +++++++++++++++++++++--
 arch/x86/include/asm/bitops.h      |   6 +-
 arch/x86/include/asm/cmpxchg.h     |  39 ++++++
 arch/x86/include/asm/hw_irq.h      |   4 +-
 arch/x86/include/asm/local.h       | 106 +++++++++++++--
 arch/x86/include/asm/preempt.h     |   2 +-
 arch/x86/include/asm/rmwcc.h       |  84 ++++++++++--
 arch/x86/include/asm/rwsem.h       |  60 ++++++++-
 arch/x86/kernel/apic/apic.c        |   2 +-
 arch/x86/kernel/apic/io_apic.c     |   4 +-
 arch/x86/kernel/cpu/mcheck/mce.c   |  12 +-
 arch/x86/kernel/i8259.c            |   2 +-
 arch/x86/kernel/irq.c              |   8 +-
 arch/x86/kernel/kgdb.c             |   6 +-
 arch/x86/kernel/pvclock.c          |   8 +-
 arch/x86/kernel/tboot.c            |   8 +-
 arch/x86/kernel/traps.c            |   6 +
 arch/x86/lib/atomic64_386_32.S     | 164 +++++++++++++++++++++++
 arch/x86/lib/atomic64_cx8_32.S     |  94 ++++++++++++-
 arch/x86/mm/mmio-mod.c             |   4 +-
 fs/exec.c                          |  23 ++++
 23 files changed, 1081 insertions(+), 88 deletions(-)
arch/x86/include/asm/atomic.h +244 -19
index e916895..0a77aa0 100644
@@ -28,6 +28,17 @@ static __always_inline int atomic_read(const atomic_t *v)
 }
 
 /**
+ * atomic_read_unchecked - read atomic variable
+ * @v: pointer of type atomic_unchecked_t
+ *
+ * Atomically reads the value of @v.
+ */
+static __always_inline int atomic_read_unchecked(const atomic_unchecked_t *v)
+{
+	return ACCESS_ONCE((v)->counter);
+}
+
+/**
  * atomic_set - set atomic variable
  * @v: pointer of type atomic_t
  * @i: required value
@@ -40,6 +51,18 @@ static __always_inline void atomic_set(atomic_t *v, int i)
 }
 
 /**
+ * atomic_set_unchecked - set atomic variable
+ * @v: pointer of type atomic_unchecked_t
+ * @i: required value
+ *
+ * Atomically sets the value of @v to @i.
+ */
+static __always_inline void atomic_set_unchecked(atomic_unchecked_t *v, int i)
+{
+	v->counter = i;
+}
+
+/**
  * atomic_add - add integer to atomic variable
  * @i: integer value to add
  * @v: pointer of type atomic_t
@@ -48,7 +71,29 @@ static __always_inline void atomic_set(atomic_t *v, int i)
  */
 static __always_inline void atomic_add(int i, atomic_t *v)
 {
-	asm volatile(LOCK_PREFIX "addl %1,%0"
+	asm volatile(LOCK_PREFIX "addl %1,%0\n"
+
+#ifdef CONFIG_PAX_REFCOUNT
+		     "jno 0f\n"
+		     LOCK_PREFIX "subl %1,%0\n"
+		     "int $4\n0:\n"
+		     _ASM_EXTABLE(0b, 0b)
+#endif
+
+		     : "+m" (v->counter)
+		     : "ir" (i));
+}
+
+/**
+ * atomic_add_unchecked - add integer to atomic variable
+ * @i: integer value to add
+ * @v: pointer of type atomic_unchecked_t
+ *
+ * Atomically adds @i to @v.
+ */
+static __always_inline void atomic_add_unchecked(int i, atomic_unchecked_t *v)
+{
+	asm volatile(LOCK_PREFIX "addl %1,%0\n"
 		     : "+m" (v->counter)
 		     : "ir" (i));
 }
@@ -62,7 +107,29 @@ static __always_inline void atomic_add(int i, atomic_t *v)
  */
 static __always_inline void atomic_sub(int i, atomic_t *v)
 {
-	asm volatile(LOCK_PREFIX "subl %1,%0"
+	asm volatile(LOCK_PREFIX "subl %1,%0\n"
+
+#ifdef CONFIG_PAX_REFCOUNT
+		     "jno 0f\n"
+		     LOCK_PREFIX "addl %1,%0\n"
+		     "int $4\n0:\n"
+		     _ASM_EXTABLE(0b, 0b)
+#endif
+
+		     : "+m" (v->counter)
+		     : "ir" (i));
+}
+
+/**
+ * atomic_sub_unchecked - subtract integer from atomic variable
+ * @i: integer value to subtract
+ * @v: pointer of type atomic_unchecked_t
+ *
+ * Atomically subtracts @i from @v.
+ */
+static __always_inline void atomic_sub_unchecked(int i, atomic_unchecked_t *v)
+{
+	asm volatile(LOCK_PREFIX "subl %1,%0\n"
 		     : "+m" (v->counter)
 		     : "ir" (i));
 }
@@ -78,7 +145,7 @@ static __always_inline void atomic_sub(int i, atomic_t *v)
  */
 static __always_inline int atomic_sub_and_test(int i, atomic_t *v)
 {
-	GEN_BINARY_RMWcc(LOCK_PREFIX "subl", v->counter, "er", i, "%0", "e");
+	GEN_BINARY_RMWcc(LOCK_PREFIX "subl", LOCK_PREFIX "addl",  v->counter, "er", i, "%0", "e");
 }
 
 /**
@@ -89,7 +156,27 @@ static __always_inline int atomic_sub_and_test(int i, atomic_t *v)
  */
 static __always_inline void atomic_inc(atomic_t *v)
 {
-	asm volatile(LOCK_PREFIX "incl %0"
+	asm volatile(LOCK_PREFIX "incl %0\n"
+
+#ifdef CONFIG_PAX_REFCOUNT
+		     "jno 0f\n"
+		     LOCK_PREFIX "decl %0\n"
+		     "int $4\n0:\n"
+		     _ASM_EXTABLE(0b, 0b)
+#endif
+
+		     : "+m" (v->counter));
+}
+
+/**
+ * atomic_inc_unchecked - increment atomic variable
+ * @v: pointer of type atomic_unchecked_t
+ *
+ * Atomically increments @v by 1.
+ */
+static __always_inline void atomic_inc_unchecked(atomic_unchecked_t *v)
+{
+	asm volatile(LOCK_PREFIX "incl %0\n"
 		     : "+m" (v->counter));
 }
 
@@ -101,7 +188,27 @@ static __always_inline void atomic_inc(atomic_t *v)
  */
 static __always_inline void atomic_dec(atomic_t *v)
 {
-	asm volatile(LOCK_PREFIX "decl %0"
+	asm volatile(LOCK_PREFIX "decl %0\n"
+
+#ifdef CONFIG_PAX_REFCOUNT
+		     "jno 0f\n"
+		     LOCK_PREFIX "incl %0\n"
+		     "int $4\n0:\n"
+		     _ASM_EXTABLE(0b, 0b)
+#endif
+
+		     : "+m" (v->counter));
+}
+
+/**
+ * atomic_dec_unchecked - decrement atomic variable
+ * @v: pointer of type atomic_unchecked_t
+ *
+ * Atomically decrements @v by 1.
+ */
+static __always_inline void atomic_dec_unchecked(atomic_unchecked_t *v)
+{
+	asm volatile(LOCK_PREFIX "decl %0\n"
 		     : "+m" (v->counter));
 }
 
@@ -115,7 +222,7 @@ static __always_inline void atomic_dec(atomic_t *v)
  */
 static __always_inline int atomic_dec_and_test(atomic_t *v)
 {
-	GEN_UNARY_RMWcc(LOCK_PREFIX "decl", v->counter, "%0", "e");
+	GEN_UNARY_RMWcc(LOCK_PREFIX "decl", LOCK_PREFIX "incl", v->counter, "%0", "e");
 }
 
 /**
@@ -128,7 +235,20 @@ static __always_inline int atomic_dec_and_test(atomic_t *v)
  */
 static __always_inline int atomic_inc_and_test(atomic_t *v)
 {
-	GEN_UNARY_RMWcc(LOCK_PREFIX "incl", v->counter, "%0", "e");
+	GEN_UNARY_RMWcc(LOCK_PREFIX "incl", LOCK_PREFIX "decl", v->counter, "%0", "e");
+}
+
+/**
+ * atomic_inc_and_test_unchecked - increment and test
+ * @v: pointer of type atomic_unchecked_t
+ *
+ * Atomically increments @v by 1
+ * and returns true if the result is zero, or false for all
+ * other cases.
+ */
+static __always_inline int atomic_inc_and_test_unchecked(atomic_unchecked_t *v)
+{
+	GEN_UNARY_RMWcc_unchecked(LOCK_PREFIX "incl", v->counter, "%0", "e");
 }
 
 /**
@@ -142,7 +262,7 @@ static __always_inline int atomic_inc_and_test(atomic_t *v)
  */
 static __always_inline int atomic_add_negative(int i, atomic_t *v)
 {
-	GEN_BINARY_RMWcc(LOCK_PREFIX "addl", v->counter, "er", i, "%0", "s");
+	GEN_BINARY_RMWcc(LOCK_PREFIX "addl", LOCK_PREFIX "subl", v->counter, "er", i, "%0", "s");
 }
 
 /**
@@ -154,6 +274,18 @@ static __always_inline int atomic_add_negative(int i, atomic_t *v)
  */
 static __always_inline int atomic_add_return(int i, atomic_t *v)
 {
+	return i + xadd_check_overflow(&v->counter, i);
+}
+
+/**
+ * atomic_add_return_unchecked - add integer and return
+ * @i: integer value to add
+ * @v: pointer of type atomi_uncheckedc_t
+ *
+ * Atomically adds @i to @v and returns @i + @v
+ */
+static __always_inline int atomic_add_return_unchecked(int i, atomic_unchecked_t *v)
+{
 	return i + xadd(&v->counter, i);
 }
 
@@ -170,6 +302,10 @@ static __always_inline int atomic_sub_return(int i, atomic_t *v)
 }
 
 #define atomic_inc_return(v)  (atomic_add_return(1, v))
+static __always_inline int atomic_inc_return_unchecked(atomic_unchecked_t *v)
+{
+	return atomic_add_return_unchecked(1, v);
+}
 #define atomic_dec_return(v)  (atomic_sub_return(1, v))
 
 static __always_inline int atomic_cmpxchg(atomic_t *v, int old, int new)
@@ -177,11 +313,21 @@ static __always_inline int atomic_cmpxchg(atomic_t *v, int old, int new)
 	return cmpxchg(&v->counter, old, new);
 }
 
+static __always_inline int atomic_cmpxchg_unchecked(atomic_unchecked_t *v, int old, int new)
+{
+	return cmpxchg(&v->counter, old, new);
+}
+
 static inline int atomic_xchg(atomic_t *v, int new)
 {
 	return xchg(&v->counter, new);
 }
 
+static inline int atomic_xchg_unchecked(atomic_unchecked_t *v, int new)
+{
+	return xchg(&v->counter, new);
+}
+
 /**
  * __atomic_add_unless - add unless the number is already a given value
  * @v: pointer of type atomic_t
@@ -193,12 +339,25 @@ static inline int atomic_xchg(atomic_t *v, int new)
  */
 static __always_inline int __atomic_add_unless(atomic_t *v, int a, int u)
 {
-	int c, old;
+	int c, old, new;
 	c = atomic_read(v);
 	for (;;) {
-		if (unlikely(c == (u)))
+		if (unlikely(c == u))
 			break;
-		old = atomic_cmpxchg((v), c, c + (a));
+
+		asm volatile("addl %2,%0\n"
+
+#ifdef CONFIG_PAX_REFCOUNT
+			     "jno 0f\n"
+			     "subl %2,%0\n"
+			     "int $4\n0:\n"
+			     _ASM_EXTABLE(0b, 0b)
+#endif
+
+			     : "=r" (new)
+			     : "0" (c), "ir" (a));
+
+		old = atomic_cmpxchg(v, c, new);
 		if (likely(old == c))
 			break;
 		c = old;
@@ -207,6 +366,49 @@ static __always_inline int __atomic_add_unless(atomic_t *v, int a, int u)
 }
 
 /**
+ * atomic_inc_not_zero_hint - increment if not null
+ * @v: pointer of type atomic_t
+ * @hint: probable value of the atomic before the increment
+ *
+ * This version of atomic_inc_not_zero() gives a hint of probable
+ * value of the atomic. This helps processor to not read the memory
+ * before doing the atomic read/modify/write cycle, lowering
+ * number of bus transactions on some arches.
+ *
+ * Returns: 0 if increment was not done, 1 otherwise.
+ */
+#define atomic_inc_not_zero_hint atomic_inc_not_zero_hint
+static inline int atomic_inc_not_zero_hint(atomic_t *v, int hint)
+{
+	int val, c = hint, new;
+
+	/* sanity test, should be removed by compiler if hint is a constant */
+	if (!hint)
+		return __atomic_add_unless(v, 1, 0);
+
+	do {
+		asm volatile("incl %0\n"
+
+#ifdef CONFIG_PAX_REFCOUNT
+			     "jno 0f\n"
+			     "decl %0\n"
+			     "int $4\n0:\n"
+			     _ASM_EXTABLE(0b, 0b)
+#endif
+
+			     : "=r" (new)
+			     : "0" (c));
+
+		val = atomic_cmpxchg(v, c, new);
+		if (val == c)
+			return 1;
+		c = val;
+	} while (c);
+
+	return 0;
+}
+
+/**
  * atomic_inc_short - increment of a short integer
  * @v: pointer to type int
  *
@@ -220,14 +422,37 @@ static __always_inline short int atomic_inc_short(short int *v)
 }
 
 /* These are x86-specific, used by some header files */
-#define atomic_clear_mask(mask, addr)				\
-	asm volatile(LOCK_PREFIX "andl %0,%1"			\
-		     : : "r" (~(mask)), "m" (*(addr)) : "memory")
-
-#define atomic_set_mask(mask, addr)				\
-	asm volatile(LOCK_PREFIX "orl %0,%1"			\
-		     : : "r" ((unsigned)(mask)), "m" (*(addr))	\
-		     : "memory")
+static inline void atomic_clear_mask(unsigned int mask, atomic_t *v)
+{
+	asm volatile(LOCK_PREFIX "andl %1,%0"
+		     : "+m" (v->counter)
+		     : "r" (~(mask))
+		     : "memory");
+}
+
+static inline void atomic_clear_mask_unchecked(unsigned int mask, atomic_unchecked_t *v)
+{
+	asm volatile(LOCK_PREFIX "andl %1,%0"
+		     : "+m" (v->counter)
+		     : "r" (~(mask))
+		     : "memory");
+}
+
+static inline void atomic_set_mask(unsigned int mask, atomic_t *v)
+{
+	asm volatile(LOCK_PREFIX "orl %1,%0"
+		     : "+m" (v->counter)
+		     : "r" (mask)
+		     : "memory");
+}
+
+static inline void atomic_set_mask_unchecked(unsigned int mask, atomic_unchecked_t *v)
+{
+	asm volatile(LOCK_PREFIX "orl %1,%0"
+		     : "+m" (v->counter)
+		     : "r" (mask)
+		     : "memory");
+}
 
 #ifdef CONFIG_X86_32
 # include <asm/atomic64_32.h>
arch/x86/include/asm/atomic64_32.h +100 -0

163 lines not on this page · raw message

arch/x86/include/asm/atomic64_64.h +154 -10

255 lines not on this page · raw message

arch/x86/include/asm/bitops.h +3 -3
index cfe3b95..69126a1 100644
@@ -203,7 +203,7 @@ static inline void change_bit(long nr, volatile unsigned long *addr)
  */
 static inline int test_and_set_bit(long nr, volatile unsigned long *addr)
 {
-	GEN_BINARY_RMWcc(LOCK_PREFIX "bts", *addr, "Ir", nr, "%0", "c");
+	GEN_BINARY_RMWcc_unchecked(LOCK_PREFIX "bts", *addr, "Ir", nr, "%0", "c");
 }
 
 /**
@@ -249,7 +249,7 @@ static inline int __test_and_set_bit(long nr, volatile unsigned long *addr)
  */
 static inline int test_and_clear_bit(long nr, volatile unsigned long *addr)
 {
-	GEN_BINARY_RMWcc(LOCK_PREFIX "btr", *addr, "Ir", nr, "%0", "c");
+	GEN_BINARY_RMWcc_unchecked(LOCK_PREFIX "btr", *addr, "Ir", nr, "%0", "c");
 }
 
 /**
@@ -302,7 +302,7 @@ static inline int __test_and_change_bit(long nr, volatile unsigned long *addr)
  */
 static inline int test_and_change_bit(long nr, volatile unsigned long *addr)
 {
-	GEN_BINARY_RMWcc(LOCK_PREFIX "btc", *addr, "Ir", nr, "%0", "c");
+	GEN_BINARY_RMWcc_unchecked(LOCK_PREFIX "btc", *addr, "Ir", nr, "%0", "c");
 }
 
 static __always_inline int constant_test_bit(long nr, const volatile unsigned long *addr)
arch/x86/include/asm/cmpxchg.h +39 -0
index ad19841..0784041 100644
@@ -14,8 +14,12 @@ extern void __cmpxchg_wrong_size(void)
 	__compiletime_error("Bad argument size for cmpxchg");
 extern void __xadd_wrong_size(void)
 	__compiletime_error("Bad argument size for xadd");
+extern void __xadd_check_overflow_wrong_size(void)
+	__compiletime_error("Bad argument size for xadd_check_overflow");
 extern void __add_wrong_size(void)
 	__compiletime_error("Bad argument size for add");
+extern void __add_check_overflow_wrong_size(void)
+	__compiletime_error("Bad argument size for add_check_overflow");
 
 /*
  * Constants for operation sizes. On 32-bit, the 64-bit size it set to
@@ -67,6 +71,38 @@ extern void __add_wrong_size(void)
 		__ret;							\
 	})
 
+#ifdef CONFIG_PAX_REFCOUNT
+#define __xchg_op_check_overflow(ptr, arg, op, lock)			\
+	({								\
+	        __typeof__ (*(ptr)) __ret = (arg);			\
+		switch (sizeof(*(ptr))) {				\
+		case __X86_CASE_L:					\
+			asm volatile (lock #op "l %0, %1\n"		\
+				      "jno 0f\n"			\
+				      "mov %0,%1\n"			\
+				      "int $4\n0:\n"			\
+				      _ASM_EXTABLE(0b, 0b)		\
+				      : "+r" (__ret), "+m" (*(ptr))	\
+				      : : "memory", "cc");		\
+			break;						\
+		case __X86_CASE_Q:					\
+			asm volatile (lock #op "q %q0, %1\n"		\
+				      "jno 0f\n"			\
+				      "mov %0,%1\n"			\
+				      "int $4\n0:\n"			\
+				      _ASM_EXTABLE(0b, 0b)		\
+				      : "+r" (__ret), "+m" (*(ptr))	\
+				      : : "memory", "cc");		\
+			break;						\
+		default:						\
+			__ ## op ## _check_overflow_wrong_size();	\
+		}							\
+		__ret;							\
+	})
+#else
+#define __xchg_op_check_overflow(ptr, arg, op, lock) __xchg_op(ptr, arg, op, lock)
+#endif
+
 /*
  * Note: no "lock" prefix even on SMP: xchg always implies lock anyway.
  * Since this is generally used to protect other memory information, we
@@ -165,6 +201,9 @@ extern void __add_wrong_size(void)
 #define xadd_sync(ptr, inc)	__xadd((ptr), (inc), "lock; ")
 #define xadd_local(ptr, inc)	__xadd((ptr), (inc), "")
 
+#define __xadd_check_overflow(ptr, inc, lock)	__xchg_op_check_overflow((ptr), (inc), xadd, lock)
+#define xadd_check_overflow(ptr, inc)		__xadd_check_overflow((ptr), (inc), LOCK_PREFIX)
+
 #define __add(ptr, inc, lock)						\
 	({								\
 	        __typeof__ (*(ptr)) __ret = (inc);			\
arch/x86/include/asm/hw_irq.h +2 -2
index 6615032..9c233be 100644
@@ -158,8 +158,8 @@ static inline void unlock_vector_lock(void) {}
 #endif	/* CONFIG_X86_LOCAL_APIC */
 
 /* Statistics */
-extern atomic_t irq_err_count;
-extern atomic_t irq_mis_count;
+extern atomic_unchecked_t irq_err_count;
+extern atomic_unchecked_t irq_mis_count;
 
 extern void elcr_set_level_irq(unsigned int irq);
 
arch/x86/include/asm/local.h +98 -8

171 lines not on this page · raw message

arch/x86/include/asm/preempt.h +1 -1
index b12f810..aedcc13 100644
@@ -84,7 +84,7 @@ static __always_inline void __preempt_count_sub(int val)
  */
 static __always_inline bool __preempt_count_dec_and_test(void)
 {
-	GEN_UNARY_RMWcc("decl", __preempt_count, __percpu_arg(0), "e");
+	GEN_UNARY_RMWcc("decl", "incl", __preempt_count, __percpu_arg(0), "e");
 }
 
 /*
arch/x86/include/asm/rmwcc.h +74 -10

107 lines not on this page · raw message

arch/x86/include/asm/rwsem.h +58 -2

108 lines not on this page · raw message

arch/x86/kernel/apic/apic.c +1 -1
index 307a498..60e3759 100644
@@ -1864,7 +1864,7 @@ static inline void __smp_error_interrupt(struct pt_regs *regs)
 		apic_write(APIC_ESR, 0);
 	v = apic_read(APIC_ESR);
 	ack_APIC_irq();
-	atomic_inc(&irq_err_count);
+	atomic_inc_unchecked(&irq_err_count);
 
 	apic_printk(APIC_DEBUG, KERN_DEBUG "APIC error on CPU%d: %02x",
 		    smp_processor_id(), v);
arch/x86/kernel/apic/io_apic.c +2 -2
index 11b46d9..473648d 100644
@@ -1682,7 +1682,7 @@ static unsigned int startup_ioapic_irq(struct irq_data *data)
 	return was_pending;
 }
 
-atomic_t irq_mis_count;
+atomic_unchecked_t irq_mis_count;
 
 #ifdef CONFIG_GENERIC_PENDING_IRQ
 static bool io_apic_level_ack_pending(struct mp_chip_data *data)
@@ -1821,7 +1821,7 @@ static void ioapic_ack_level(struct irq_data *irq_data)
 	 * at the cpu.
 	 */
 	if (!(v & (1 << (i & 0x1f)))) {
-		atomic_inc(&irq_mis_count);
+		atomic_inc_unchecked(&irq_mis_count);
 		eoi_ioapic_pin(cfg->vector, irq_data->chip_data);
 	}
 
arch/x86/kernel/cpu/mcheck/mce.c +6 -6
index df919ff..22f7362 100644
@@ -292,10 +292,10 @@ static void print_mce(struct mce *m)
 
 #define PANIC_TIMEOUT 5 /* 5 seconds */
 
-static atomic_t mce_panicked;
+static atomic_unchecked_t mce_panicked;
 
 static int fake_panic;
-static atomic_t mce_fake_panicked;
+static atomic_unchecked_t mce_fake_panicked;
 
 /* Panic in progress. Enable interrupts and wait for final IPI */
 static void wait_for_panic(void)
@@ -319,7 +319,7 @@ static void mce_panic(const char *msg, struct mce *final, char *exp)
 		/*
 		 * Make sure only one CPU runs in machine check panic
 		 */
-		if (atomic_inc_return(&mce_panicked) > 1)
+		if (atomic_inc_return_unchecked(&mce_panicked) > 1)
 			wait_for_panic();
 		barrier();
 
@@ -327,7 +327,7 @@ static void mce_panic(const char *msg, struct mce *final, char *exp)
 		console_verbose();
 	} else {
 		/* Don't log too much for fake panic */
-		if (atomic_inc_return(&mce_fake_panicked) > 1)
+		if (atomic_inc_return_unchecked(&mce_fake_panicked) > 1)
 			return;
 	}
 	/* First print corrected ones that are still unlogged */
@@ -752,7 +752,7 @@ static int mce_timed_out(u64 *t, const char *msg)
 	 * might have been modified by someone else.
 	 */
 	rmb();
-	if (atomic_read(&mce_panicked))
+	if (atomic_read_unchecked(&mce_panicked))
 		wait_for_panic();
 	if (!mca_cfg.monarch_timeout)
 		goto out;
@@ -2555,7 +2555,7 @@ struct dentry *mce_get_debugfs_dir(void)
 static void mce_reset(void)
 {
 	cpu_missing = 0;
-	atomic_set(&mce_fake_panicked, 0);
+	atomic_set_unchecked(&mce_fake_panicked, 0);
 	atomic_set(&mce_executing, 0);
 	atomic_set(&mce_callin, 0);
 	atomic_set(&global_nwo, 0);
arch/x86/kernel/i8259.c +1 -1
index 16cb827..989dbb2 100644
@@ -208,7 +208,7 @@ spurious_8259A_irq:
 			       "spurious 8259A interrupt: IRQ%d.\n", irq);
 			spurious_irq_mask |= irqmask;
 		}
-		atomic_inc(&irq_err_count);
+		atomic_inc_unchecked(&irq_err_count);
 		/*
 		 * Theoretically we do not have to handle this IRQ,
 		 * but in Linux this does not cause problems and is
arch/x86/kernel/irq.c +4 -4
index c7dfe1b..146f63c 100644
@@ -28,7 +28,7 @@ EXPORT_PER_CPU_SYMBOL(irq_stat);
 DEFINE_PER_CPU(struct pt_regs *, irq_regs);
 EXPORT_PER_CPU_SYMBOL(irq_regs);
 
-atomic_t irq_err_count;
+atomic_unchecked_t irq_err_count;
 
 /* Function pointer for generic interrupt vector handling */
 void (*x86_platform_ipi_callback)(void) = NULL;
@@ -144,9 +144,9 @@ int arch_show_interrupts(struct seq_file *p, int prec)
 		seq_printf(p, "%10u ", irq_stats(j)->irq_hv_callback_count);
 	seq_puts(p, "  Hypervisor callback interrupts\n");
 #endif
-	seq_printf(p, "%*s: %10u\n", prec, "ERR", atomic_read(&irq_err_count));
+	seq_printf(p, "%*s: %10u\n", prec, "ERR", atomic_read_unchecked(&irq_err_count));
 #if defined(CONFIG_X86_IO_APIC)
-	seq_printf(p, "%*s: %10u\n", prec, "MIS", atomic_read(&irq_mis_count));
+	seq_printf(p, "%*s: %10u\n", prec, "MIS", atomic_read_unchecked(&irq_mis_count));
 #endif
 #ifdef CONFIG_HAVE_KVM
 	seq_printf(p, "%*s: ", prec, "PIN");
@@ -198,7 +198,7 @@ u64 arch_irq_stat_cpu(unsigned int cpu)
 
 u64 arch_irq_stat(void)
 {
-	u64 sum = atomic_read(&irq_err_count);
+	u64 sum = atomic_read_unchecked(&irq_err_count);
 	return sum;
 }
 
arch/x86/kernel/kgdb.c +3 -3
index d6178d9..cb20db8 100644
@@ -475,12 +475,12 @@ int kgdb_arch_handle_exception(int e_vector, int signo, int err_code,
 	case 'k':
 		/* clear the trace bit */
 		linux_regs->flags &= ~X86_EFLAGS_TF;
-		atomic_set(&kgdb_cpu_doing_single_step, -1);
+		atomic_set_unchecked(&kgdb_cpu_doing_single_step, -1);
 
 		/* set the trace bit if we're stepping */
 		if (remcomInBuffer[0] == 's') {
 			linux_regs->flags |= X86_EFLAGS_TF;
-			atomic_set(&kgdb_cpu_doing_single_step,
+			atomic_set_unchecked(&kgdb_cpu_doing_single_step,
 				   raw_smp_processor_id());
 		}
 
@@ -545,7 +545,7 @@ static int __kgdb_notify(struct die_args *args, unsigned long cmd)
 
 	switch (cmd) {
 	case DIE_DEBUG:
-		if (atomic_read(&kgdb_cpu_doing_single_step) != -1) {
+		if (atomic_read_unchecked(&kgdb_cpu_doing_single_step) != -1) {
 			if (user_mode(regs))
 				return single_step_cont(regs, args);
 			break;
arch/x86/kernel/pvclock.c +4 -4
index 2f355d2..e75ed0a 100644
@@ -51,11 +51,11 @@ void pvclock_touch_watchdogs(void)
 	reset_hung_task_detector();
 }
 
-static atomic64_t last_value = ATOMIC64_INIT(0);
+static atomic64_unchecked_t last_value = ATOMIC64_INIT(0);
 
 void pvclock_resume(void)
 {
-	atomic64_set(&last_value, 0);
+	atomic64_set_unchecked(&last_value, 0);
 }
 
 u8 pvclock_read_flags(struct pvclock_vcpu_time_info *src)
@@ -105,11 +105,11 @@ cycle_t pvclock_clocksource_read(struct pvclock_vcpu_time_info *src)
 	 * updating at the same time, and one of them could be slightly behind,
 	 * making the assumption that last_value always go forward fail to hold.
 	 */
-	last = atomic64_read(&last_value);
+	last = atomic64_read_unchecked(&last_value);
 	do {
 		if (ret < last)
 			return last;
-		last = atomic64_cmpxchg(&last_value, last, ret);
+		last = atomic64_cmpxchg_unchecked(&last_value, last, ret);
 	} while (unlikely(last != ret));
 
 	return ret;
arch/x86/kernel/tboot.c +4 -4
index 91a4496..d1a52ee 100644
@@ -310,7 +310,7 @@ static int tboot_extended_sleep(u8 sleep_state, u32 val_a, u32 val_b)
 	return -ENODEV;
 }
 
-static atomic_t ap_wfs_count;
+static atomic_unchecked_t ap_wfs_count;
 
 static int tboot_wait_for_aps(int num_aps)
 {
@@ -334,9 +334,9 @@ static int tboot_cpu_callback(struct notifier_block *nfb, unsigned long action,
 {
 	switch (action) {
 	case CPU_DYING:
-		atomic_inc(&ap_wfs_count);
+		atomic_inc_unchecked(&ap_wfs_count);
 		if (num_online_cpus() == 1)
-			if (tboot_wait_for_aps(atomic_read(&ap_wfs_count)))
+			if (tboot_wait_for_aps(atomic_read_unchecked(&ap_wfs_count)))
 				return NOTIFY_BAD;
 		break;
 	}
@@ -422,7 +422,7 @@ static __init int tboot_late_init(void)
 
 	tboot_create_trampoline();
 
-	atomic_set(&ap_wfs_count, 0);
+	atomic_set_unchecked(&ap_wfs_count, 0);
 	register_hotcpu_notifier(&tboot_cpu_notifier);
 
 #ifdef CONFIG_DEBUG_FS
arch/x86/kernel/traps.c +6 -0
index f579192..4328936 100644
@@ -213,6 +213,12 @@ do_trap_no_signal(struct task_struct *tsk, int trapnr, char *str,
 			tsk->thread.trap_nr = trapnr;
 			die(str, regs, error_code);
 		}
+
+#ifdef CONFIG_PAX_REFCOUNT
+		if (trapnr == X86_TRAP_OF)
+			pax_report_refcount_overflow(regs);
+#endif
+
 		return 0;
 	}
 
arch/x86/lib/atomic64_386_32.S +164 -0

242 lines not on this page · raw message

arch/x86/lib/atomic64_cx8_32.S +88 -6

196 lines not on this page · raw message

arch/x86/mm/mmio-mod.c +2 -2
index 0057a7a..6fa3eb7 100644
@@ -234,7 +234,7 @@ static void post(struct kmmio_probe *p, unsigned long condition,
 static void ioremap_trace_core(resource_size_t offset, unsigned long size,
 							void __iomem *addr)
 {
-	static atomic_t next_id;
+	static atomic_unchecked_t next_id;
 	struct remap_trace *trace = kmalloc(sizeof(*trace), GFP_KERNEL);
 	/* These are page-unaligned. */
 	struct mmiotrace_map map = {
@@ -258,7 +258,7 @@ static void ioremap_trace_core(resource_size_t offset, unsigned long size,
 			.private = trace
 		},
 		.phys = offset,
-		.id = atomic_inc_return(&next_id)
+		.id = atomic_inc_return_unchecked(&next_id)
 	};
 	map.map_id = trace->id;
 
fs/exec.c +23 -0
index 1977c2a..e9552b2 100644
@@ -57,6 +57,11 @@
 #include <linux/oom.h>
 #include <linux/compat.h>
 
+#ifdef CONFIG_PAX_REFCOUNT
+#include <linux/kallsyms.h>
+#include <linux/kdebug.h>
+#endif
+
 #include <asm/uaccess.h>
 #include <asm/mmu_context.h>
 #include <asm/tlb.h>
@@ -1743,3 +1748,21 @@ COMPAT_SYSCALL_DEFINE5(execveat, int, fd,
 				  argv, envp, flags);
 }
 #endif
+
+#ifdef CONFIG_PAX_REFCOUNT
+void pax_report_refcount_overflow(struct pt_regs *regs)
+{
+	if (current->signal->curr_ip)
+		printk(KERN_EMERG "PAX: From %pI4: refcount overflow detected in: %s:%d, uid/euid: %u/%u\n",
+				&current->signal->curr_ip, current->comm, task_pid_nr(current),
+				from_kuid_munged(&init_user_ns, current_uid()), from_kuid_munged(&init_user_ns, current_euid()));
+	else
+		printk(KERN_EMERG "PAX: refcount overflow detected in: %s:%d, uid/euid: %u/%u\n", current->comm, task_pid_nr(current),
+				from_kuid_munged(&init_user_ns, current_uid()), from_kuid_munged(&init_user_ns, current_euid()));
+	print_symbol(KERN_EMERG "PAX: refcount overflow occured at: %s\n", instruction_pointer(regs));
+	preempt_disable();
+	show_regs(regs);
+	preempt_enable();
+	force_sig_info(SIGKILL, SEND_SIG_FORCED, current);
+}
+#endif
Signature · 1 line
2.5.0
#2 lore raw

RFC PATCH 2/5 percpu_ref: decrease per-CPU refcount bias

In reply to David Windsor

This change is necessary to prevent overflows from occuring in percpu_switch_to_atomic_rcu during init on x86.

---
 lib/percpu-refcount.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/percpu-refcount.c +1 -1
index 6111bcb..02e816b 100644
@@ -31,7 +31,7 @@
  * atomic_long_t can't hit 0 before we've added up all the percpu refs.
  */
 
-#define PERCPU_COUNT_BIAS	(1LU << (BITS_PER_LONG - 1))
+#define PERCPU_COUNT_BIAS	(1LU << (BITS_PER_LONG - 2))
 
 static DECLARE_WAIT_QUEUE_HEAD(percpu_ref_switch_waitq);
 
Signature · 1 line
2.5.0
#3 lore raw

RFC PATCH 4/5 security: add Kconfig options for PAX_REFCOUNT

In reply to David Windsor

Currently the Kconfig options for this feature retain the original naming from Pax/grsecurity.

---
 security/Kconfig | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 72 insertions(+)
security/Kconfig +72 -0
index bf4ec46..cf08034 100644
@@ -4,6 +4,78 @@
 
 menu "Security options"
 
+menu "Grsecurity"
+
+config GRKERNSEC
+	bool "Grsecurity"
+	help
+	  If you say Y here, you will be able to configure many features
+	  that will enhance the security of your system.  It is highly
+	  recommended that you say Y here and read through the help
+	  for each option so that you fully understand the features and
+	  can evaluate their usefulness for your machine.
+
+choice
+	prompt "Configuration Method"
+	depends on GRKERNSEC
+	default GRKERNSEC_CONFIG_CUSTOM
+	help
+
+config GRKERNSEC_CONFIG_AUTO
+	bool "Automatic"
+	help
+	  If you choose this configuration method, you'll be able to answer a small
+	  number of simple questions about how you plan to use this kernel.
+	  The settings of grsecurity and PaX will be automatically configured for
+	  the highest commonly-used settings within the provided constraints.
+
+	  If you require additional configuration, custom changes can still be made
+	  from the "custom configuration" menu.
+
+config GRKERNSEC_CONFIG_CUSTOM
+	bool "Custom"
+	help
+	  If you choose this configuration method, you'll be able to configure all
+	  grsecurity and PaX settings manually.  Via this method, no options are
+	  automatically enabled.
+
+	  Take note that if menuconfig is exited with this configuration method
+	  chosen, you will not be able to use the automatic configuration methods
+	  without starting again with a kernel configuration with no grsecurity
+	  or PaX options specified inside.
+
+endchoice
+
+menu "PaX"
+
+menu "Miscellaneous hardening features"
+
+config PAX_REFCOUNT
+	bool "Prevent various kernel object reference counter overflows"
+	default y if GRKERNSEC_CONFIG_AUTO
+	depends on GRKERNSEC && ((ARM && (CPU_V6 || CPU_V6K || CPU_V7)) || MIPS || PPC || SPARC64 || X86)
+	help
+	  By saying Y here the kernel will detect and prevent overflowing
+	  various (but not all) kinds of object reference counters.  Such
+	  overflows can normally occur due to bugs only and are often, if
+	  not always, exploitable.
+
+	  The tradeoff is that data structures protected by an overflowed
+	  refcount will never be freed and therefore will leak memory.  Note
+	  that this leak also happens even without this protection but in
+	  that case the overflow can eventually trigger the freeing of the
+	  data structure while it is still being used elsewhere, resulting
+	  in the exploitable situation that this feature prevents.
+
+	  Since this has a negligible performance impact, you should enable
+	  this feature.
+
+endmenu
+
+endmenu
+
+endmenu
+
 source security/keys/Kconfig
 
 config SECURITY_DMESG_RESTRICT
Signature · 1 line
2.5.0
#4 lore raw

RFC PATCH 5/5 lkdtm: add test for atomic_t underflow/overflow

In reply to David Windsor

dmesg output of running this LKDTM test:

[187095.475573] lkdtm: No crash points registered, enable through debugfs
[187118.020257] lkdtm: Performing direct entry WRAP_ATOMIC
[187118.030045] lkdtm: attempting atomic underflow
[187118.030929] PAX: refcount overflow detected in: bash:1790, uid/euid: 0/0
[187118.071667] PAX: refcount overflow occured at: lkdtm_do_action+0x19e/0x400 [lkdtm]
[187118.081423] CPU: 3 PID: 1790 Comm: bash Not tainted 4.2.6-pax-refcount-split+ #2
[187118.083403] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
[187118.102596] task: ffff8800da8de040 ti: ffff8800da8e4000 task.ti: ffff8800da8e4000
[187118.111321] RIP: 0010:[<ffffffffc00fc2fe>]  [<ffffffffc00fc2fe>] lkdtm_do_action+0x19e/0x400 [lkdtm]
...
[187118.128074] lkdtm: attempting atomic overflow
[187118.128080] PAX: refcount overflow detected in: bash:1790, uid/euid: 0/0
[187118.128082] PAX: refcount overflow occured at: lkdtm_do_action+0x1b6/0x400 [lkdtm]
[187118.128085] CPU: 3 PID: 1790 Comm: bash Not tainted 4.2.6-pax-refcount-split+ #2
[187118.128086] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
[187118.128088] task: ffff8800da8de040 ti: ffff8800da8e4000 task.ti: ffff8800da8e4000
[187118.128092] RIP: 0010:[<ffffffffc00fc316>]  [<ffffffffc00fc316>] lkdtm_do_action+0x1b6/0x400 [lkdtm]
---
 drivers/misc/lkdtm.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
drivers/misc/lkdtm.c +13 -0
index b5abe34..5002747 100644
@@ -104,6 +104,7 @@ enum ctype {
 	CT_ACCESS_USERSPACE,
 	CT_WRITE_RO,
 	CT_WRITE_KERN,
+    CT_WRAP_ATOMIC
 };
 
 static char* cp_name[] = {
@@ -141,6 +142,7 @@ static char* cp_type[] = {
 	"ACCESS_USERSPACE",
 	"WRITE_RO",
 	"WRITE_KERN",
+    "WRAP_ATOMIC"
 };
 
 static struct jprobe lkdtm;
@@ -522,6 +524,17 @@ static void lkdtm_do_action(enum ctype which)
 		do_overwritten();
 		break;
 	}
+    case CT_WRAP_ATOMIC: {
+        atomic_t under = ATOMIC_INIT(INT_MIN);
+        atomic_t over = ATOMIC_INIT(INT_MAX);
+
+        pr_info("attempting atomic underflow\n");
+        atomic_dec(&under);
+        pr_info("attempting atomic overflow\n");
+        atomic_inc(&over);
+
+        return;
+    }
 	case CT_NONE:
 	default:
 		break;
Signature · 1 line
2.5.0
#5 lore raw

Re: [kernel-hardening][RFC PATCH 5/5] lkdtm: add test for atomic_t underflow/overflow

drivers/misc/lkdtm.c enum ctype { · line 110

Quoted 12 lines · David Windsor
drivers/misc/lkdtm.c +1 -0
index b5abe34..5002747 100644
@@ -104,6 +104,7 @@ enum ctype {
 	CT_ACCESS_USERSPACE,
 	CT_WRITE_RO,
 	CT_WRITE_KERN,
+    CT_WRAP_ATOMIC
 };
 
 static char* cp_name[] = {

You seem to be indenting with spaces rather than tabs. It'd probably be a good idea to run checkpatch on the patches. It should catch this.

David