Memory: Operations

Mem Alloc

  • This function allocates size  bytes of memory, aligned to a boundary specified by alignment  using the allocator specified by allocator .

  • If the size  parameter is 0 , the operation is a no-op.

  • Inputs :

    • size : The desired size of the allocated memory region.

    • alignment : The desired alignment of the allocated memory region.

    • allocator : The allocator to allocate from.

  • core:mem

@(require_results)
alloc :: proc(
    size: int,
    alignment: int = DEFAULT_ALIGNMENT,
    allocator := context.allocator,
    loc := #caller_location,
) -> (rawptr, Allocator_Error) {
    data, err := runtime.mem_alloc(size, alignment, allocator, loc)
    return raw_data(data), err
}
@(require_results)
alloc_bytes :: proc(
    size: int,
    alignment: int = DEFAULT_ALIGNMENT,
    allocator := context.allocator,
    loc := #caller_location,
) -> ([]byte, Allocator_Error) {
    return runtime.mem_alloc(size, alignment, allocator, loc)
}
@(require_results)
alloc_bytes_non_zeroed :: proc(
    size: int,
    alignment: int = DEFAULT_ALIGNMENT,
    allocator := context.allocator,
    loc := #caller_location,
) -> ([]byte, Allocator_Error) {
    return runtime.mem_alloc_non_zeroed(size, alignment, allocator, loc)
}
  • base:runtime

mem_alloc :: #force_no_inline proc(size: int, alignment: int = DEFAULT_ALIGNMENT, allocator := context.allocator, loc := #caller_location) -> ([]byte, Allocator_Error) {
    assert(is_power_of_two_int(alignment), "Alignment must be a power of two", loc)
    if size == 0 || allocator.procedure == nil {
        return nil, nil
    }
    return allocator.procedure(allocator.data, .Alloc, size, alignment, nil, 0, loc)
}
mem_alloc_bytes :: #force_no_inline proc(size: int, alignment: int = DEFAULT_ALIGNMENT, allocator := context.allocator, loc := #caller_location) -> ([]byte, Allocator_Error) {
    assert(is_power_of_two_int(alignment), "Alignment must be a power of two", loc)
    if size == 0 || allocator.procedure == nil{
        return nil, nil
    }
    return allocator.procedure(allocator.data, .Alloc, size, alignment, nil, 0, loc)
}

New

  • Allocates a single object.

  • Returns a pointer to a newly allocated value of that type using the specified allocator.

  • base:builtin

new
@(builtin, require_results)
new :: proc($T: typeid, allocator := context.allocator, loc := #caller_location) -> (t: ^T, err: Allocator_Error) #optional_allocator_error {
    t = (^T)(raw_data(mem_alloc_bytes(size_of(T), align_of(T), allocator, loc) or_return))
    return
}
  • Example :

    ptr := new(int)
    ptr^ = 123
    x: int = ptr^
    
new_aligned
@(require_results)
new_aligned :: proc($T: typeid, alignment: int, allocator := context.allocator, loc := #caller_location) -> (t: ^T, err: Allocator_Error) {
    t = (^T)(raw_data(mem_alloc_bytes(size_of(T), alignment, allocator, loc) or_return))
    return
}
new_clone
  • Allocates a clone of the value  passed to it.

  • The resulting value of the type will be a pointer  to the type of the value passed.

@(builtin, require_results)
new_clone :: proc(data: $T, allocator := context.allocator, loc := #caller_location) -> (t: ^T, err: Allocator_Error) #optional_allocator_error {
    t = (^T)(raw_data(mem_alloc_bytes(size_of(T), align_of(T), allocator, loc) or_return))
    if t != nil {
        t^ = data
    }
    return
}
  • Example :

    ptr: ^int = new_clone(123)
    assert(ptr^ == 123)
    

Mem Free

  • Free a single object (opposite of new )

  • Will try to free the passed pointer, with the given allocator  if the allocator supports this operation.

  • Only free memory with the allocator it was allocated with.

Cautions
  • Trying to free an object that is "zero-initialized" will not cause a "bad-free".

  • free(&...)  will almost always be wrong, or at best unnecessary.

    • If you need to use &  to get a pointer to something, then that something probably isn't allocated at all.

    • If it were allocated, you'd already have a pointer. e.g., in that example, free(&d)  would be trying to free a pointer to the stack--that'll never end well.

    • For built-in types like slices, dynamic arrays, maps, and strings, use delete  for those instead. They're not pointers themselves, but they have a pointer internally.

Procs
  • base:builtin

@builtin
free :: proc{mem_free}
  • base:runtime

mem_free :: #force_no_inline proc(ptr: rawptr, allocator := context.allocator, loc := #caller_location) -> Allocator_Error {
    if ptr == nil || allocator.procedure == nil {
        return nil
    }
    _, err := allocator.procedure(allocator.data, .Free, 0, 0, ptr, 0, loc)
    return err
}

mem_free_with_size :: #force_no_inline proc(ptr: rawptr, byte_count: int, allocator := context.allocator, loc := #caller_location) -> Allocator_Error {
    if ptr == nil || allocator.procedure == nil {
        return nil
    }
    _, err := allocator.procedure(allocator.data, .Free, 0, 0, ptr, byte_count, loc)
    return err
}

mem_free_bytes :: #force_no_inline proc(bytes: []byte, allocator := context.allocator, loc := #caller_location) -> Allocator_Error {
    if bytes == nil || allocator.procedure == nil {
        return nil
    }
    _, err := allocator.procedure(allocator.data, .Free, 0, 0, raw_data(bytes), len(bytes), loc)
    return err
}
  • core:mem

free :: proc(
    ptr: rawptr,
    allocator := context.allocator,
    loc := #caller_location,
) -> Allocator_Error {
    return runtime.mem_free(ptr, allocator, loc)
}

Mem Free All

  • Will try to free/reset all of the memory of the given allocator  if the allocator supports this operation.

  • base:builtin

// `free_all` will try to free/reset all of the memory of the given `allocator` if the allocator supports this operation.
@builtin
free_all :: proc{mem_free_all}
  • base:runtime

mem_free_all :: #force_no_inline proc(allocator := context.allocator, loc := #caller_location) -> (err: Allocator_Error) {
    if allocator.procedure != nil {
        _, err = allocator.procedure(allocator.data, .Free_All, 0, 0, nil, 0, loc)
    }
    return
}

Make

  • Allocates and initializes a value of type slice, dynamic array, map, or multi-pointer (only).

  • Unlike new , make 's return value is the same as the type of its argument, not a pointer to it.

  • Like new , the first argument is a type, not a value.

  • Uses the specified allocator; the default is context.allocator .

  • base:builtin

@builtin
make :: proc{
    make_slice,
    make_dynamic_array,
    make_dynamic_array_len,
    make_dynamic_array_len_cap,
    make_map,
    make_map_cap,
    make_multi_pointer,
    make_soa_slice,
    make_soa_dynamic_array,
    make_soa_dynamic_array_len,
    make_soa_dynamic_array_len_cap,
}
  • make_aligned

    • Not included in make .

  • make_soa_aligned

    • Not included in make .

Examples
slice := make([]int, 65)

dynamic_array_zero_length := make([dynamic]int)
dynamic_array_with_length := make([dynamic]int, 32)
dynamic_array_with_length_and_capacity := make([dynamic]int, 16, 64)

made_map := make(map[string]int)
made_map_with_reservation := make(map[string]int, 64)
Allocation in structs
  • Caio:

    • How can I ensure that the [dynamic]  arrays in the struct physics_packet  below use a custom allocator?

    Physics_Packet :: struct {
        tick_number: u64,
        data: Physics_Data,
    }
    
    Physics_Data :: struct {
        characters: [dynamic]Character_Data,
        creatures: [dynamic]Creature_Data,
    }
    
    physics_packet: Physics_Packet
    
    • When doing something like

    append(&physics_packet.data.characters, {             
        id = personagem.socket,            
        pos = personagem.pos        
    })
    
    
    • How do I not use the context.allocator  but a custom allocator? I created the struct just by doing physics_packet: Physics_Packet , so which allocator is used to define the arrays characters  and creatures ?

  • Barinzaya:

    • Use physics_packet.data.characters = make([dynamic]Character_Data, allocator=custom_alloc)

  • Chamberlain:

    • The first append actually does the allocation unless you do it explicitly with make.

  • Caio:

    • What about ZII? Shouldn't I delete the "previous array" before assigning it with a make ? Or even, is there a way that I define the whole struct using a custom allocator, without having to redefine the arrays, just to use a different allocator? There's also the question of which allocator is used when creating the struct.

  • Chamberlain:

    • No allocator is used when creating the struct.

  • Barinzaya:

    • If you declared physics_packet  as a local variable, then it'll be zero-initialized on the stack. No allocation will happen (depending on how technical you are about "allocation"; technically it was allocated on the stack--but that's completely unrelated to Allocator s). That includes the dynamic arrays in it, where "zero" means "no pointer, 0 length, no allocator".

slice
  • Allocates and initializes a slice

@(require_results)
make_aligned :: proc($T: typeid/[]$E, #any_int len: int, alignment: int, allocator := context.allocator, loc := #caller_location) -> (res: T, err: Allocator_Error) #optional_allocator_error {
    err = _make_aligned_type_erased(&res, size_of(E), len, alignment, allocator, loc)
    return
}

@(require_results)
_make_aligned_type_erased :: proc(slice: rawptr, elem_size: int, len: int, alignment: int, allocator: Allocator, loc := #caller_location) -> Allocator_Error {
    make_slice_error_loc(loc, len)
    data, err := mem_alloc_bytes(elem_size*len, alignment, allocator, loc)
    if data == nil && elem_size != 0 {
        return err
    }
    (^Raw_Slice)(slice).data = raw_data(data)
    (^Raw_Slice)(slice).len  = len
    return err
}

@(builtin, require_results)
make_slice :: proc($T: typeid/[]$E, #any_int len: int, allocator := context.allocator, loc := #caller_location) -> (res: T, err: Allocator_Error) #optional_allocator_error {
    err = _make_aligned_type_erased(&res, size_of(E), len, align_of(E), allocator, loc)
    return
}
dynamic array
  • Allocates and initializes a dynamic array.

@(builtin, require_results)
make_dynamic_array :: proc($T: typeid/[dynamic]$E, allocator := context.allocator, loc := #caller_location) -> (array: T, err: Allocator_Error) #optional_allocator_error {
    err = _make_dynamic_array_len_cap((^Raw_Dynamic_Array)(&array), size_of(E), align_of(E), 0, 0, allocator, loc)
    return
}

@(builtin, require_results)
make_dynamic_array_len :: proc($T: typeid/[dynamic]$E, #any_int len: int, allocator := context.allocator, loc := #caller_location) -> (array: T, err: Allocator_Error) #optional_allocator_error {
    err = _make_dynamic_array_len_cap((^Raw_Dynamic_Array)(&array), size_of(E), align_of(E), len, len, allocator, loc)
    return
}

@(builtin, require_results)
make_dynamic_array_len_cap :: proc($T: typeid/[dynamic]$E, #any_int len: int, #any_int cap: int, allocator := context.allocator, loc := #caller_location) -> (array: T, err: Allocator_Error) #optional_allocator_error {
    err = _make_dynamic_array_len_cap((^Raw_Dynamic_Array)(&array), size_of(E), align_of(E), len, cap, allocator, loc)
    return
}

@(require_results)
_make_dynamic_array_len_cap :: proc(array: ^Raw_Dynamic_Array, size_of_elem, align_of_elem: int, #any_int len: int, #any_int cap: int, allocator := context.allocator, loc := #caller_location) -> (err: Allocator_Error) {
    make_dynamic_array_error_loc(loc, len, cap)
    array.allocator = allocator // initialize allocator before just in case it fails to allocate any memory
    data := mem_alloc_bytes(size_of_elem*cap, align_of_elem, allocator, loc) or_return
    use_zero := data == nil && size_of_elem != 0
    array.data = raw_data(data)
    array.len = 0 if use_zero else len
    array.cap = 0 if use_zero else cap
    array.allocator = allocator
    return
}
map
  • Initializes a map with an allocator.

@(builtin, require_results)
make_map :: proc($T: typeid/map[$K]$E, allocator := context.allocator, loc := #caller_location) -> (m: T) {
    m.allocator = allocator
    return m
}

@(builtin, require_results)
make_map_cap :: proc($T: typeid/map[$K]$E, #any_int capacity: int, allocator := context.allocator, loc := #caller_location) -> (m: T, err: Allocator_Error) #optional_allocator_error {
    make_map_expr_error_loc(loc, capacity)
    context.allocator = allocator
    err = reserve_map(&m, capacity, loc)
    return
}
Multi-pointer
  • Allocates and initializes a multi-pointer.

@(builtin, require_results)
make_multi_pointer :: proc($T: typeid/[^]$E, #any_int len: int, allocator := context.allocator, loc := #caller_location) -> (mp: T, err: Allocator_Error) #optional_allocator_error {
    make_slice_error_loc(loc, len)
    data := mem_alloc_bytes(size_of(E)*len, align_of(E), allocator, loc) or_return
    if data == nil && size_of(E) != 0 {
        return
    }
    mp = cast(T)raw_data(data)
    return
}

Deletes

  • Free a group of objects (opposite of make )

  • Deletes the backing memory of a value allocated with make or a string that was allocated through an allocator.

  • Will try to free the underlying data of the passed built-in data structure (string, cstring, dynamic array, slice, or map), with the given allocator  if the allocator supports this operation.

  • base:builtin

@builtin
delete :: proc{
    delete_string,
    delete_cstring,
    delete_dynamic_array,
    delete_slice,
    delete_map,
    delete_soa_slice,
    delete_soa_dynamic_array,
    delete_string16,
    delete_cstring16,
}
  • Recursiveness :

    • delete  isn't recursive. It has no way of knowing whether you actually want  to delete the contents or not--you may not always.

        array_args_as_bytes: [dynamic][]u8
    
        // Option 1: Don't delete everything.
        defer delete(array_args_as_bytes)
    
        // Option 2: Delete everything.
        defer {
            for arg in array_args_as_bytes {
                delete(arg)
            }
            delete(array_args_as_bytes)
        }
    
    • If it's a struct, it's not uncommon to make a destroy_struct  proc that does this for you.

    • The way I understand it is that the data  of the object is deleted. The data  itself is a pointer to where the data is stored, so deleting the data  is deleting the pointer.

string
@builtin
delete_string :: proc(str: string, allocator := context.allocator, loc := #caller_location) -> Allocator_Error {
    return mem_free_with_size(raw_data(str), len(str), allocator, loc)
}
cstring
@builtin
delete_cstring :: proc(str: cstring, allocator := context.allocator, loc := #caller_location) -> Allocator_Error {
    return mem_free((^byte)(str), allocator, loc)
}
string16
@builtin
delete_string16 :: proc(str: string16, allocator := context.allocator, loc := #caller_location) -> Allocator_Error {
    return mem_free_with_size(raw_data(str), len(str)*size_of(u16), allocator, loc)
}
cstring16
@builtin
delete_cstring16 :: proc(str: cstring16, allocator := context.allocator, loc := #caller_location) -> Allocator_Error {
    return mem_free((^u16)(str), allocator, loc)
}
dynamic array
@builtin
delete_dynamic_array :: proc(array: $T/[dynamic]$E, loc := #caller_location) -> Allocator_Error {
    return mem_free_with_size(raw_data(array), cap(array)*size_of(E), array.allocator, loc)
}
slice
@builtin
delete_slice :: proc(array: $T/[]$E, allocator := context.allocator, loc := #caller_location) -> Allocator_Error {
    return mem_free_with_size(raw_data(array), len(array)*size_of(E), allocator, loc)
}
Map
@builtin
delete_map :: proc(m: $T/map[$K]$V, loc := #caller_location) -> Allocator_Error {
    return map_free_dynamic(transmute(Raw_Map)m, map_info(T), loc)
}

Mem Resize

  • base:runtime

_mem_resize :: #force_no_inline proc(
    ptr: rawptr, 
    old_size, 
    new_size: int, 
    alignment: int = DEFAULT_ALIGNMENT, 
    allocator := context.allocator, 
    should_zero: bool, 
    loc := #caller_location
    ) -> (data: []byte, err: Allocator_Error) {
    
    assert(is_power_of_two_int(alignment), "Alignment must be a power of two", loc)
    if allocator.procedure == nil {
        return nil, nil
    }
    if new_size == 0 {
        if ptr != nil {
            _, err = allocator.procedure(allocator.data, .Free, 0, 0, ptr, old_size, loc)
            return
        }
        return
    } else if ptr == nil {
        if should_zero {
            return allocator.procedure(allocator.data, .Alloc, new_size, alignment, nil, 0, loc)
        } else {
            return allocator.procedure(allocator.data, .Alloc_Non_Zeroed, new_size, alignment, nil, 0, loc)
        }
    } else if old_size == new_size && uintptr(ptr) % uintptr(alignment) == 0 {
        data = ([^]byte)(ptr)[:old_size]
        return
    }
    if should_zero {
        data, err = allocator.procedure(allocator.data, .Resize, new_size, alignment, ptr, old_size, loc)
    } else {
        data, err = allocator.procedure(allocator.data, .Resize_Non_Zeroed, new_size, alignment, ptr, old_size, loc)
    }
    if err == .Mode_Not_Implemented {
        if should_zero {
            data, err = allocator.procedure(allocator.data, .Alloc, new_size, alignment, nil, 0, loc)
        } else {
            data, err = allocator.procedure(allocator.data, .Alloc_Non_Zeroed, new_size, alignment, nil, 0, loc)
        }
        if err != nil {
            return
        }
        copy(data, ([^]byte)(ptr)[:old_size])
        _, err = allocator.procedure(allocator.data, .Free, 0, 0, ptr, old_size, loc)
    }
    return
}

mem_resize :: proc(
    ptr: rawptr, 
    old_size, 
    new_size: int, 
    alignment: int = DEFAULT_ALIGNMENT, 
    allocator := context.allocator, 
    loc := #caller_location
    ) -> (data: []byte, err: Allocator_Error) {
    assert(is_power_of_two_int(alignment), "Alignment must be a power of two", loc)
    return _mem_resize(ptr, old_size, new_size, alignment, allocator, true, loc)
}

non_zero_mem_resize :: proc(
    ptr: rawptr, 
    old_size, 
    new_size: int, 
    alignment: int = DEFAULT_ALIGNMENT, 
    allocator := context.allocator, 
    loc := #caller_location
    ) -> (data: []byte, err: Allocator_Error) {
    assert(is_power_of_two_int(alignment), "Alignment must be a power of two", loc)
    return _mem_resize(ptr, old_size, new_size, alignment, allocator, false, loc)
}

Mem Set

  • Set a number of bytes ( len ) to a value ( val ), from the address specified ( ptr ).

Using the 'C Runtime Library' (CRT)
  • base:runtime

when ODIN_NO_CRT == true && ODIN_OS == .Windows {
    @(link_name="memset", linkage="strong", require)
    memset :: proc "c" (ptr: rawptr, val: i32, len: int) -> rawptr {
        RtlFillMemory(ptr, len, val)
        return ptr
    }
} else when ODIN_NO_CRT || (ODIN_OS != .Orca && (ODIN_ARCH == .wasm32 || ODIN_ARCH == .wasm64p32)) {
    @(link_name="memset", linkage="strong", require)
    memset :: proc "c" (ptr: rawptr, val: i32, #any_int len: int_t) -> rawptr {
        if ptr != nil && len != 0 {
            b := byte(val)
            p := ([^]byte)(ptr)
            for i := int_t(0); i < len; i += 1 {
                p[i] = b
            }
        }
        return ptr
    }
} else {
    memset :: proc "c" (ptr: rawptr, val: i32, len: int) -> rawptr {
        if ptr != nil && len != 0 {
            b := byte(val)
            p := ([^]byte)(ptr)
            for i := 0; i < len; i += 1 {
                p[i] = b
            }
        }
        return ptr
    }
}
In C

Mem Copy

Which one to use
  • TLDR :

    • Barinzaya / Tetralux / Yawning:

      • Use copy .

      • The difference in performance is going to be pretty small between any of them. Anything non_overlapping  is a slight optimization at most if you know for sure  it won't overlap. If it does, it may completely wreck your data.

      • Use intrinsics.mem_copy_non_overlapping  or other option if you profile and see copy  to be an issue.

  • copy

    • For convenience and safety, but slower.

  • intrinsics.mem_copy_non_overlapping

    • For speed and no safety.

  • runtime.copy  / runtime.copy_non_overlapping

    • A middle ground between the two above, I guess,

  • mem.copy  / mem.copy_non_overlapping

    • Just a indirection from intrinsics.mem_copy  / intrinsics.mem_copy_non_overlapping .

    • mem.copy  is a tiny wrapper that will almost certainly end up inlined with any optimization on.

  • core:c/libc

    • Ignore this one, is just there for completeness.

Equivalence to C's
  • mem_copy

    • Similar to C's memmove .

    • Requires a little bit of additional logic to correctly handle the ranges overlapping.

  • mem_copy_non_overlapping

    • Similar to C's memcopy .

Using intrinsics
  • Barinzaya:

    • The intrinsic  is handled by the compiler. It does a bit of additional "smart" stuff--if the length is constant, it emits the instructions to do the copy inline (without a call), and it just tells LLVM to do the memcpy / memmove . LLVM may in fact just call the memcpy / memmove  proc (provided by the CRT or procs.odin ), if it sees fit.

    • But it still allows LLVM to be a little "smarter" about it, AFAIK. Since it knows  what the proc does, it can potentially elide the copy (though probably less so in the case where the length is variable).

    • Every available copy procedure uses intrinsics.mem_copy  or intrinsics.mem_copy_non_overlapping  under the hood, so therefore, all those implementations benefit from possible compiler optimizations.

  • base:runtime

    • Builtin.

    • Slice / Strings.

    • Copies elements from a source slice/string src  to a destination slice dst .

    • The source and destination may overlap. Copy returns the number of elements copied, which will be the minimum of len(src)  and len(dst) .

    @(require_results)
    copy_slice_raw :: proc "contextless" (dst, src: rawptr, dst_len, src_len, elem_size: int) -> int {
        n := min(dst_len, src_len)
        if n > 0 {
            intrinsics.mem_copy(dst, src, n*elem_size)
        }
        return n
    }
    
    @builtin
    copy_slice :: #force_inline proc "contextless" (dst, src: $T/[]$E) -> int {
        return copy_slice_raw(raw_data(dst), raw_data(src), len(dst), len(src), size_of(E))
    }
    
    @builtin
    copy_from_string :: #force_inline proc "contextless" (dst: $T/[]$E/u8, src: $S/string) -> int {
        return copy_slice_raw(raw_data(dst), raw_data(src), len(dst), len(src), 1)
    }
    
    @builtin
    copy :: proc{copy_slice, copy_from_string, copy_from_string16}
    
  • base:runtime

    • General.

    mem_copy :: proc "contextless" (dst, src: rawptr, len: int) -> rawptr {
        if src != nil && dst != src && len > 0 {
            // NOTE(bill): This _must_ be implemented like C's memmove
            intrinsics.mem_copy(dst, src, len)
        }
        return dst
    }
    
    mem_copy_non_overlapping :: proc "contextless" (dst, src: rawptr, len: int) -> rawptr {
        if src != nil && dst != src && len > 0 {
            // NOTE(bill): This _must_ be implemented like C's memcpy
            intrinsics.mem_copy_non_overlapping(dst, src, len)
        }
        return dst
    }
    
  • core:mem

    copy :: proc "contextless" (dst, src: rawptr, len: int) -> rawptr {
        intrinsics.mem_copy(dst, src, len)
        return dst
    }
    
    copy_non_overlapping :: proc "contextless" (dst, src: rawptr, len: int) -> rawptr {
        intrinsics.mem_copy_non_overlapping(dst, src, len)
        return dst
    }
    
  • base:intrinsics

    mem_copy                 :: proc(dst, src: rawptr, len: int) ---
    mem_copy_non_overlapping :: proc(dst, src: rawptr, len: int) ---
    
Using core:c/libc
  • Barinzaya:

    • It's just procs from libc--part of which is the CRT. So the libc  one is explicitly the CRT implementation.

  • core:c/libc

memcpy   :: proc(s1, s2: rawptr, n: size_t) -> rawptr ---

memmove  :: proc(s1, s2: rawptr, n: size_t) -> rawptr ---

strcpy   :: proc(s1: [^]char, s2: cstring) -> [^]char ---
strncpy  :: proc(s1: [^]char, s2: cstring, n: size_t) -> [^]char ---
Implementation from 'C Runtime Library' (CRT)
  • ODIN_NO_CRT .

    • true  if the -no-crt  command line switch is passed, which inhibits linking with the C Runtime Library, a.k.a. LibC.

    • The default is false , so CRT is  used.

  • Should I enabled CRT or not? I forgot to ask that, oops

  • Barinzaya:

    • memcpy  and memmove  are part of the C run-time, and LLVM needs to have them . If you disable the CRT, then they need to be provided--hence, why they're in procs.odin . Note that they're in when ODIN_NO_CRT  blocks (plus other conditions). So the procs.odin  implementation is used when the CRT isn't linked, because they need to exist

  • Caio:

    • Can I say that procs.odin  provides an implementation for   intrinsics  copy procedures, considering the conditions defined in the procs.odin ? As a fallback I mean, I assume intrinsics  already have an implementation somewhere.

  • Barinzaya:

    • Not entirely, procs.odin  is more "stuff needed for LLVM to work at all when the CRT isn't included"

    • intrinsics  are all implemented in the compiler itself. In the case of the copy s, they defer to LLVM intrinsics, which may  call memcpy / memmove  from the CRT or procs.odin --but they also may not

    • Also, LLVM can call memcpy / memmove   without  those intrinsics too, for sufficiently large copies.

  • Tetralux:

    • Intrinsics are more "compiler hooks" for "I want to do this thing please"

    • They are somewhat opaque things if you see what I mean

  • base:runtime

when ODIN_NO_CRT == true && ODIN_OS == .Windows {
    @(link_name="memcpy", linkage="strong", require)
    memcpy :: proc "c" (dst, src: rawptr, len: int) -> rawptr {
        RtlMoveMemory(dst, src, len)
        return dst
    }
    
    @(link_name="memmove", linkage="strong", require)
    memmove :: proc "c" (dst, src: rawptr, len: int) -> rawptr {
        RtlMoveMemory(dst, src, len)
        return dst
    }
} else when ODIN_NO_CRT || (ODIN_OS != .Orca && (ODIN_ARCH == .wasm32 || ODIN_ARCH == .wasm64p32)) {
    @(link_name="memcpy", linkage="strong", require)
    memcpy :: proc "c" (dst, src: rawptr, #any_int len: int_t) -> rawptr {
        d, s := ([^]byte)(dst), ([^]byte)(src)
        if d != s {
            for i := int_t(0); i < len; i += 1 {
                d[i] = s[i]
            }
        }
        return d
    }
    
    @(link_name="memmove", linkage="strong", require)
    memmove :: proc "c" (dst, src: rawptr, #any_int len: int_t) -> rawptr {
        d, s := ([^]byte)(dst), ([^]byte)(src)
        if d == s || len == 0 {
            return dst
        }
        if d > s && uintptr(d)-uintptr(s) < uintptr(len) {
            for i := len-1; i >= 0; i -= 1 {
                d[i] = s[i]
            }
            return dst
        }
        if s > d && uintptr(s)-uintptr(d) < uintptr(len) {
            for i := int_t(0); i < len; i += 1 {
                d[i] = s[i]
            }
            return dst
        }
        return memcpy(dst, src, len)
    }
} else {
    // None.
}
In C

Mem Zero

Using intrinsics
  • base:runtime

mem_zero :: proc "contextless" (data: rawptr, len: int) -> rawptr {
    if data == nil {
        return nil
    }
    if len <= 0 {
        return data
    }
    intrinsics.mem_zero(data, len)
    return data
}
  • base:intrinsics

mem_zero                 :: proc(ptr: rawptr, len: int) ---
mem_zero_volatile        :: proc(ptr: rawptr, len: int) ---
Conditionally Mem Zero
  • When acquiring memory from the OS for the first time it's likely that the OS already gives the zero page mapped multiple times for the request. The actual allocation does not have physical pages allocated to it until those pages are written to which causes a page-fault. This is often called COW (Copy on Write) .

  • You do not want to actually zero out memory in this case because it would cause a bunch of page faults decreasing the speed of allocations and increase the amount of actual resident physical memory used.

  • Instead a better technique is to check if memory is zerored before zeroing it. This turns out to be an important optimization in practice, saving nearly half (or more) the amount of physical memory used by an application.

  • This is why every implementation of calloc  in libc  does this optimization.

  • It may seem counter-intuitive but most allocations in an application are wasted and never used. When you consider something like a [dynamic]T  which always doubles in capacity on resize but you rarely ever actually use the full capacity of a dynamic array it means you have a lot of resident waste if you actually zeroed the remainder of the memory.

  • Keep in mind the OS is already guaranteed to give you zeroed memory by mapping in this zero page multiple times so in the best case there is no need to actually zero anything. As for testing all this memory for a zero value, it costs nothing because the the same zero page is used for the whole allocation and will exist in L1 cache for the entire zero checking process.

  • base:runtime

conditional_mem_zero :: proc "contextless" (data: rawptr, n_: int) #no_bounds_check {
    if n_ <= 0 {
        return
    }
    n := uint(n_)
    n_words := n / size_of(uintptr)
    p_words := ([^]uintptr)(data)[:n_words]
    p_bytes := ([^]byte)(data)[size_of(uintptr) * n_words:n]
    for &p_word in p_words {
        if p_word != 0 {
            p_word = 0
        }
    }
    for &p_byte in p_bytes {
        if p_byte != 0 {
            p_byte = 0
        }
    }
}
Using the 'C Runtime Library' (CRT)
when ODIN_NO_CRT && ODIN_OS == .Windows {
    // None
} else when ODIN_NO_CRT || (ODIN_OS != .Orca && (ODIN_ARCH == .wasm32 || ODIN_ARCH == .wasm64p32)) {
    @(link_name="bzero", linkage="strong", require)
    bzero :: proc "c" (ptr: rawptr, #any_int len: int_t) -> rawptr {
        if ptr != nil && len != 0 {
            p := ([^]byte)(ptr)
            for i := int_t(0); i < len; i += 1 {
                p[i] = 0
            }
        }
        return ptr
    }
 } else {
     // None
 }
In C

Resize

_mem_resize :: #force_no_inline proc(ptr: rawptr, old_size, new_size: int, alignment: int = DEFAULT_ALIGNMENT, allocator := context.allocator, should_zero: bool, loc := #caller_location) -> (data: []byte, err: Allocator_Error) {
    assert(is_power_of_two_int(alignment), "Alignment must be a power of two", loc)
    if allocator.procedure == nil {
        return nil, nil
    }
    if new_size == 0 {
        if ptr != nil {
            _, err = allocator.procedure(allocator.data, .Free, 0, 0, ptr, old_size, loc)
            return
        }
        return
    } else if ptr == nil {
        if should_zero {
            return allocator.procedure(allocator.data, .Alloc, new_size, alignment, nil, 0, loc)
        } else {
            return allocator.procedure(allocator.data, .Alloc_Non_Zeroed, new_size, alignment, nil, 0, loc)
        }
    } else if old_size == new_size && uintptr(ptr) % uintptr(alignment) == 0 {
        data = ([^]byte)(ptr)[:old_size]
        return
    }
    if should_zero {
        data, err = allocator.procedure(allocator.data, .Resize, new_size, alignment, ptr, old_size, loc)
    } else {
        data, err = allocator.procedure(allocator.data, .Resize_Non_Zeroed, new_size, alignment, ptr, old_size, loc)
    }
    if err == .Mode_Not_Implemented {
        if should_zero {
            data, err = allocator.procedure(allocator.data, .Alloc, new_size, alignment, nil, 0, loc)
        } else {
            data, err = allocator.procedure(allocator.data, .Alloc_Non_Zeroed, new_size, alignment, nil, 0, loc)
        }
        if err != nil {
            return
        }
        copy(data, ([^]byte)(ptr)[:old_size])
        _, err = allocator.procedure(allocator.data, .Free, 0, 0, ptr, old_size, loc)
    }
    return
}

mem_resize :: proc(ptr: rawptr, old_size, new_size: int, alignment: int = DEFAULT_ALIGNMENT, allocator := context.allocator, loc := #caller_location) -> (data: []byte, err: Allocator_Error) {
    assert(is_power_of_two_int(alignment), "Alignment must be a power of two", loc)
    return _mem_resize(ptr, old_size, new_size, alignment, allocator, true, loc)
}

non_zero_mem_resize :: proc(ptr: rawptr, old_size, new_size: int, alignment: int = DEFAULT_ALIGNMENT, allocator := context.allocator, loc := #caller_location) -> (data: []byte, err: Allocator_Error) {
    assert(is_power_of_two_int(alignment), "Alignment must be a power of two", loc)
    return _mem_resize(ptr, old_size, new_size, alignment, allocator, false, loc)
}
Default resize procedure
  • When allocator does not support resize operation, but supports .Alloc  / .Alloc_Non_Zeroed  and .Free , this procedure is used to implement allocator's default behavior on resize.

  • The behavior of the function is as follows:

    • If new_size  is 0 , the function acts like free() , freeing the memory region specified by old_data .

    • If old_data  is nil , the function acts like alloc() , allocating new_size  bytes of memory aligned on a boundary specified by alignment .

    • Otherwise, a new memory region of size new_size  is allocated, then the data from the old memory region is copied and the old memory region is freed.

@(require_results)
_default_resize_bytes_align :: #force_inline proc(
    old_data: []byte,
    new_size: int,
    alignment: int,
    should_zero: bool,
    allocator := context.allocator,
    loc := #caller_location,
) -> ([]byte, Allocator_Error) {
    old_memory := raw_data(old_data)
    old_size := len(old_data)
    if old_memory == nil {
        if should_zero {
            return alloc_bytes(new_size, alignment, allocator, loc)
        } else {
            return alloc_bytes_non_zeroed(new_size, alignment, allocator, loc)
        }
    }
    if new_size == 0 {
        err := free_bytes(old_data, allocator, loc)
        return nil, err
    }
    if new_size == old_size && is_aligned(old_memory, alignment) {
        return old_data, .None
    }
    new_memory : []byte
    err : Allocator_Error
    if should_zero {
        new_memory, err = alloc_bytes(new_size, alignment, allocator, loc)
    } else {
        new_memory, err = alloc_bytes_non_zeroed(new_size, alignment, allocator, loc)
    }
    if new_memory == nil || err != nil {
        return nil, err
    }
    runtime.copy(new_memory, old_data)
    free_bytes(old_data, allocator, loc)
    return new_memory, err
}