Sub module's auxiliary stack boundary and bottom may be different from
main module's counterpart, so when calling sub module, its aux stack info
should be gotten and set to exec_env firstly, or aux stack overflow and out
of bounds memory access exception may be thrown when calling sub
module's function.
Fix the issue reported in PR #1278.
Enable aot compiler and jit based on llvm-14.0 and llvm-15.0git,
replace LLVMBuildLoad/LLVMBuildInBoundsGEP/LLVMBuildCall with
LLVMBuildLoad2/LLVMBuildInBoundsGEP2/LLVMBuildCall2, and pass
them with related types, so as to meet the requirements of opaque
pointers.
And fix several compilation errors for llvm-14.0/15.0git.
Most spec cases and standalone cases are tested.
Fix the following warning introduced by the recent change.
```
In file included from wamr/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/locking.h:17,
from wamr/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/posix.h:18,
from wamr/core/iwasm/aot/../common/wasm_runtime_common.h:18,
from wamr/core/iwasm/aot/aot_runtime.h:10,
from wamr/core/iwasm/aot/aot_loader.c:6:
wamr/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/ssp_config.h:55: error: "CONFIG_HAS_ISATTY" redefined [-Werror]
#define CONFIG_HAS_ISATTY 1
In file included from wamr/core/shared/utils/../platform/include/platform_common.h:13,
from wamr/core/shared/utils/bh_platform.h:9,
from wamr/core/iwasm/aot/aot_runtime.h:9,
from wamr/core/iwasm/aot/aot_loader.c:6:
wamr/core/shared/platform/nuttx/platform_internal.h:75: note: this is the location of the previous definition
#define CONFIG_HAS_ISATTY 0
```
When using clang compiler, the f32/f64 return value might be
invalid when calling invokeNative asm code. Declare the return
type of invokeNative as void, and set volatile for the converted
function pointers to resolve the issue.
Enable dump call stack to a buffer, use API
`wasm_runtime_get_call_stack_buf_size` to get the required buffer size
and use API
`wasm_runtime_dump_call_stack_to_buf` to dump call stack to a buffer
Support integrating 3rd-party toolchain llc compiler or asm compiler
into wamrc by setting environment variable WAMRC_LLC_COMPILER
or WAMRC_ASM_COMPILER, wamrc will use these tools to generate
object file from LLVM IR firstly, and then refactor the object file into
aot file.
Implement Go binding APIs of runtime, module and instance
Add sample, build scripts and update the document
Co-authored-by: venus-taibai <97893654+venus-taibai@users.noreply.github.com>
This header file is supposed to be used by user code, which is not
a part of WAMR. Usually WAMR configuration is not available there,
remove DEBUG_INTERP macro control in it.
Re-implement socket api send/recv in an atomic-like way, fix the return value
check in posix.c.
And fix wasm-c-api sample callback_chain calling malloc issue.
Automatically dump memory/performance profiling data in
wasm_application_execute_main and wasm_application_execute_func when
the related feature is enabled.
And remove unused aot_compile_wasm_file func declaration in aot_compiler.h.
wasm_c_api.c: add more checks, fix LOG_WARNING invalid specifier
aot_emit_aot_file: fix strncpy max size length to copy
posix.c: fix potential socket not close issue
wasm-c-api samples: add return value checks for fseek/ftell
cJSON.c: remove dead code
module_wasm_app.c: add return value check for wasm_runtime_call_wasm
aot_runtime.c: add return value check for aot_get_default_memory
aot_runtime.c: add return value check before calling wasm app malloc/free func
wasm_runtime_common.c: fix dead code warning in wasm_runtime_load_from_sections
aot_emit_memory.c: fix potential integer overflow issue
wasm_runtime.c: remove dead code in memory_instantiate, add assertion for globals
samples simple/gui/littlevgl: fix fields of struct sigaction initialization issue
host-tool: add return value check for sendto
Fix the symbol resolving failure with recent version of wamrc:
```
AOT module load failed: resolve symbol .Lswitch.table.aot _func#82.2 failed
```
Replace the relocations for such symbols with .rodata section.
Add assertion for BH_MALLOC/BH_FREE in wasm_runtime_common.c,
when building runtime, the BH_MALLOC/BH_FREE macros should be
defined as wasm_runtime_malloc/wasm_runtime_free.
Fix allocate zero size memory warning reported by wasm_runtime_malloc
when allocating the import fun pointers if the import func count is 0:
`warning: wasm_runtime_malloc with size zero`
Fix wasm_application_execute_main/wasm_application_execute_func not waiting for
other threads to terminate in multi-thread mode, which causes that the exception
thrown by other threads may haven't been spreaded to current main thread, and
cannot be detected by the caller, as reported in #1131.
Refine opcode br_table for classic interpreter as there may be a lot of
leb128 decoding when the br count is big:
1. Use the bytecode itself to store the decoded leb br depths if each
decoded depth can be stored with one byte
2. Create br_table cache to store the decode leb br depths if the decoded
depth cannot be stored with one byte
After the optimization, the class interpreter can access the br depths array
with index, no need to decode the leb128 again.
And fix function record_fast_op() return value unchecked issue in source
debugging feature.
Fix fault modification on data segment in wasm module which leads to
fail to instantiate wasm module next time, reported by #1115.
Co-authored-by: yangwenming <yangwenming@bytedance.com>
Fix an UBSan complaint introduced by recent change by adding more checks
to word_copy:
```
wasm_interp_fast.c:792:9: runtime error: applying zero offset to null pointer
```
Fix the following warning when loading an aot file without relocations:
```
[20:19:00:528 - 1119F1600]: warning: wasm_runtime_malloc with size zero
```
Fix issues in PR "Refine interp/aot string storage and emitting (#820)",
which had a few issues:
- It looks a wrong byte to mark the flag
- It doesn't work for long strings (>= 0x80 in case of little endian)
This commit fixes them by maintaining a list of loaded symbols while loading
relocation section to avoid reading a string repeatedly, and no need to mark
the flag again.
Add aot relocation for ".rodata.str" symbol to support more cases
Fix some coding style issues
Fix aot block/value stack destroy issue
Refine classic/fast interpreter codes
Clear compile warning of libc_builtin_wrapper.c in 32-bit platform
Fix compile warnings in libc-wasi posix.c:
```
posix.c:880:41: warning: comparison of integers of different signs:
'unsigned long' and 'ssize_t' (aka 'long') [-Wsign-compare]
if (bufoff + iov[i].buf_len < len) {
posix.c:1359:32: warning: comparison of integers of different signs:
'off_t' (aka 'long long') and 'unsigned long long' [-Wsign-compare]
if (ret == 0 && sb.st_size < offset + len)
```
The format specifiers of %ld/%lu/%li/%lx/%lp/%lo (or %z* and %t* like) should
be treated as 32-bit integer in wasm.
And merge the handle of unsigned/signed for %u/%d and %llu/%lld specifier
when calling snprintf.
During debugging, the debug client may request to malloc a memory space
to evaluate the user expressions. If we malloc memory from the linear memory,
it may fail when the thread is in stop status. We preserve a buffer during
creating debug instance, and use a simple bump pointer allocator to serve lldb's
memory request.
Fix the potential integer overflow of const index in const space of fast interpreter,
emit i32/i64.const opcode when the const index is larger than INT32_MAX.
And add check for the function local cell num.
Fix handle OP_TABLE_COPY issue
Fix loader handle OP_BLOCK/IF/LOOP issue if type_index is larger than 256
Fix loader handle OP_GET_GLOBAL, allow to change to GET_GLOBAL_64 for
aot compiler similiar to handling OP_SET_GLOBAL
Refine loader handle OP_GET/SET/TEE_LOCAL, disable changing opcode when
source debugging is enabled, so as no need to record the change of opcode
Refine wasm_interp_interp_frame_size to reduce the wasm operand stack usage
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Remove the `const` flag for the first argument `buf` of wasm_runtime_load as
it might be modified by runtime for footprint and performance purpose, and
update the related functions and document.
This patch allows safer (note: safer, not safe) embedding in a plugin
environment where multiple instances of the engine could be needed.
Original code initializes and tears down the full runtime during
wasm_engine_new() and wasm_engine_delete() respectively. After this
update the C API implementation keeps track of engine instances count
and inits/deinits the runtime only when needed.
This allows for example to call wasm_engine_new() twice and then call
wasm_engine_delete() once without rendering the first engine instance
invalid.
In thread_manager.c, `wasm_cluster_spawn_exec_env` creates a new module
instance but not sets wasi_ctx, so when the new exec_env calls wasm function
which uses WASI API, the WASI API functions in `libc_wasi_wrapper.c` will get
null result in calling `get_wasi_ctx` and then return `wasi_errno`.
Signed-off-by: HangedFish <bravohangedman@outlook.com>
Refer to [Networking API design](https://github.com/WebAssembly/WASI/issues/370)
and [feat(socket): berkeley socket API v2](https://github.com/WebAssembly/WASI/pull/459):
- Support the socket API of synchronous mode, including `socket/bind/listen/accept/send/recv/close/shutdown`,
the asynchronous mode isn't supported yet.
- Support adding `--addr-pool=<pool1,pool2,..>` argument for command line to identify the valid ip address range
- Add socket-api sample and update the document
Fix several issues in wasm-c-api wasm_module_imports function:
1. Two of the if branches never set the module_name and name fields which are later passed as arguments to wasm_importtype_new, and eventually might cause double-free and/or use-after-free
2. Should zero module_name/name/extern_type at the start of loop iteration, and destroy their resources when failed at the end of loop iteration
2. No need to check `if (!extern_type) { continue; }`, as extern_type is converted from type and type is already checked
3. No need to wasm_importtype_vec_delete(out) when failed, as it is passed from outside and should be destroyed by outside
When calling native function from AOT code, current implementation is to return
back to runtime to call aot_invoke_native, which calls wasm_runtime_invoke_native
and the latter calls assembly code. We did it before as there may be pointer and
string arguments to check and convert if the native function's registered signature
has character '*' and '$'.
As the built-in native function's signatures can be gotten in compilation time, we
check the pointer/string arguments and convert them into native address in AOT
code, and then invoke the native function directly, so as to improve performance.
Implement pthread_cond_broadcast wrapper for lib-pthread
- support pthread_cond_broadcast wrapper for posix/linux-sgx/windows
- update document for building multi-thread wasm app with emcc
Use LLVM new pass manager for wamrc to replace the legacy pass manger,
so as to gain better performance and reduce the compilation time.
Reference links:
- https://llvm.org/docs/NewPassManager.html
- https://blog.llvm.org/posts/2021-03-26-the-new-pass-manager
And add an option to use the legacy pm mode when building wamrc:
cmake .. -DWAMR_BUILD_LLVM_LEGACY_PM=1
For JIT mode, keep it unchanged as it only runs several function passes and
using new pass manager will increase the compilation time.
And refactor the codes of applying LLVM passes.
Refactor LLVM Orc JIT to actually enable the lazy compilation and speedup
the launching process:
https://llvm.org/docs/ORCv2.html#laziness
Main modifications:
- Create LLVM module for each wasm function, wrap it with thread safe module
so that the modules can be compiled parallelly
- Lookup function from aot module instance's func_ptrs but not directly call the
function to decouple the module relationship
- Compile the function when it is first called and hasn't been compiled
- Create threads to pre-compile the WASM functions parallelly when loading
- Set Lazy JIT as default, update document and build/test scripts
Currently when calling wasm_runtime_call_wasm() to invoke wasm function
with externref type argument from runtime embedder, developer needs to
use wasm_externref_obj2ref() to convert externref obj into an internal ref
index firstly, which is not convenient to developer.
To align with GC feature in which all the references passed to
wasm_runtime_call_wasm() can be object pointers directly, we change the
interface of wasm_runtime_call_wasm() to allow to pass object pointer
directly for the externref argument, and refactor the related codes, update
the related samples and the document.
Add log info for heap and stack like wasm loader in aot loader
Signed-off-by: Huang Qi <huangqi3@xiaomi.com>
Change-Id: I349848d75f1a26cde29217c14cfb6e779c976a8b
Refine is_xip_file check, when e_type isn't E_TYPE_XIP, just return false
and no need to go through all the other sections of the AOT file.
Refine pointer range check, convert pointer to uintptr_t type before
comparison to yield possible sanitizer pointer overflow error.
Fix __wasi_subscription_t structure definition inconsistent with wasi-libc
definition issue, reported by #961, tested with sleep, poll API and other
wasi cases on x86-64, x86-32 and arm32 targets.
Fix the issue that joining a detached thread might result in joining hang,
resolve the issue by adding wait_count for a thread's exec_env to indicate
whether a thread needs to detach itself or not when it exits.
And add checks for the input exec_env for cluster's join/detach/cancel thread.
Put Vectorize passes before GVN/LICM passes as normally the former
gains more performance improvement and the latter might break the
optimizations for the former. Can improve performance of several
sightglass cases.
And don't check exception throw after calling an AOT function if it is
and recursive call, similar to handing of Spec tail call opcode.
Enable running XIP file on Windows platform.
And add more strict checks for wamrc to report error when the input file
is same with output file, or the input file is AOT file but not wasm file.
Fix some issues on MacOS platform
- Enable libc-wasi by default
- Set target abi to "gnu" if it is not set for wamrc to avoid generating
object file of unsupported Mach-O format
- Set `<vendor>-<sys>` info according to target abi for wamrc to support
generating AOT file for other OSs but not current host
- Set cpu name if arch/abi/cpu are not set to avoid checking SIMD
capability failed
- Set size level to 1 for MacOS/Windows platform to avoid relocation type
unsupported warning
- Clear posix_memmap.c compiling warning
- Fix spec case test script issues, enable test spec cases on MacOS
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Various fixes and beautifications coordinated with @1c3t3a,
fixes 2 of the 3 all remaining issues from #892:
- enable to os_mmap executable memory
- fix os_malloc/os_realloc/os_free issues
- implement os_thread_get_stack_boundary
- add build scripts to include with esp-idf to use wamr as
an ESP-IDF component
- update sample and document
Refer to https://github.com/WebAssembly/WASI/blob/main/design/application-abi.md
to check the WASI ABI compatibility:
- Command (main module) may export _start function with signature "()"
- Reactor (sub module) may export _initialize function with signature "()"
- _start and _initialize can not be exported at the same time
- Reactor cannot export _start function
- Command and Reactor must export memory
And
- Rename module->is_wasi_module to module->import_wasi_api
- Refactor wasm_loader_find_export()
- Remove MULTI_MODULE related codes from mini_loader
- Update multi-module samples
- Fix a "use-after-free" issue. Since we reuse the memory instance of sub module,
just to protect it from freeing an imported memory instance
- use platform independent data types in debug-engine library
- add os_socket APIs and provide windows and posix implementation
- avoid using platform related header files in non-platform layer
- use format specifiers macros for sprintf and sscanf
- change thread handle type from uint64 to korp_tid
- add lock when sending socket packet to avoid thread racing
Use `PRIxxx` related macros to format the output strings so as to clear
compile warnings, e.g. PRIu32, PRId32, PRIX32, PRIX64 and so on.
And add the related macro definitions in platform_common.h if they
are not defined, as some compilers might not support them.
Don't use constant float table on arm and riscv as LLVM doesn't generate
.LPCI/.rodata like relocations on them, the float/double constants are encoded
into instructions directly, so no need to lookup them from constant table.
Signed-off-by: Huang Qi <huangqi3@xiaomi.com>
Lookup float/double constants from exec_env->native_symbol table
but not construct them with LLVMBuildConst if XIP mode is enabled,
these constants are introduced by f32/f64.const opcodes and some
float/double conversion opcodes, and make wamrc generate some
relocations in text section of AOT XIP file. This patch eliminates such
relocations when "--enable-indirect-mode" is added to wamrc.
Update spec test cases to commit 2460ad02b51fb5ed5824f44de287a8638b19a5f8,
and modify wamr test suite script as the SIMD cases have been added into spec
main repo by default, no need to clone SIMD repo again when testing SIMD.
Use the previous resolved binary type info (obj_data->target_info.bin_type) to
check the endian and bit-width but not the raw binary info, the latter is not
suitable for the check for Win32 object file type.
And fix the symbol comparison issue in resolve_target_sym(), as in Win32, the
symbol name of a function added by LLVMAddFunction() is prefixed by '_',
which leads to invalid result returned by strcmp().
Refine some codes in wasm loader
Add -Wshadow to gcc compile flags and fix some variable shadowed issues
Fix function parameter/return types not checked issue
Fix fast-interp loader reserve_block_ret() not handle V128 return type issue
Fix mini loader load_table_segment_section() failed issue
Add detailed comments for argc argument in wasm_runtime_call_wasm()
Treat R_AARCH64_JUMP26 same as R_AARCH64_CALL26, both of their
relocation operation is S + A - P, the difference is that one is for call
(BL) instructions, the other is for jump (B) instructions.
Auto detect whether file is XIP file before loading module in posix like and
linux-sgx platforms, and if yes, mmap executable memory automatically to
run the XIP file.
Add document about XIP feature.
Enable test spec cases with XIP feature.
- move the wait_cond from exec_env to debug_instance, so the debug thread can be waken up by any threads
- process more general query message from debugger
- refine debug instance create/destroy mechanism
- avoid creating debug instance during module instantiating
- avoid blocking execution thread during creating debug instance
- update related documents
Don't throw exception when module_malloc memory failed:
- Exception will terminate the wasm app, it's not necessary since app can
check the result of dynamic allocation and do some cleanup or fallback
operation on failure instead of 'crash' directly.
- In acquire_wait_info, call hasn_map_find only when the address isn't NULL,
or there are many senseless error logs
When auxiliary stack global isn't found in wasm app, it must be unused in
the wasm app, we set it to __heap_base global and set its size to 0, so as to
shrink the linear memory to reduce the memory consumption.
The debug_instance_list is not protected by mutex, and the debug instance uses
the length of this list as its id. If several threads create debug instance at the
same time, they may get duplicated/same ids. This patch adds mutex to protect
this list.
The destroy_callback_list variable in thread_manager_destroy is not cleared.
When wamr is started as a task, after exiting and restarting, it will loop
endlessly in traverse_list.
Signed-off-by: zhouliang3 <zhouliang3@xiaomi.com>
Enable ref types feature and bulk memory feature by default for wamrc
and provide "--disable-ref-types", "--disable-bulk-memory" to disable
them.
And remove the ref_type_flag option in wasm_loader.c which is used to
control whether to enable ref types or not when ENABLE_REF_TYPES
macro is enabled in wamrc. As the wasm binary format with ref types
is compatible with the binary format before, we can remove the option.
Also update the spec test scripts.
Fix some potential pointer overflows in aot applying relocations and
several other places.
And add sanitizer compiler flags to wamrc CMakeLists.txt to detect
such issues.
Enable LLVM link time optimization for AOT and enable it by default,
and provide "wamrc --disable-llvm-lto" option to disable it.
And modify the spec test script, disable it by default when testing
spec cases.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Fix fail to get builtin functions in indirect call (XIP) mode, e.g.
function aot_alloc_frame, aot_memory_init, aot_table_init and so on.
Signed-off-by: Huang Qi <huangqi3@xiaomi.com>
Currently the string in the wasm/aot file will be duplicated and stored
into const string list/set in interpreter/aot loader, which leads to extra
unnecessary memory consumption if the file buffer can be referred to
after loading. We refine the string storage by:
- if the file buffer can be referred to after loading and it is writable, we
reuse the file buffer to store the string but not store it into the const
string set: move string backward and append '\0'
- emit string with '\0' only for XIP mode in which the AOT file is readonly
- if the file buffer cannot be referred to, e.g. in app manager, keep the
same behavior as before
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Enable emitting custom name section to aot file when adding
`--enable-dump-call-stack` or `--enable-dump-call-stack` to
wamrc and there is custom name section in wasm file, which
can be generated by wasi-sdk/emcc "-g" option. So aot runtime
can also get the function name from the custom name section
instead of export section, to which developer should use
`--export-all` for wasi-sdk/emcc to generate export function
names.
Apply clang-format for C source files in folder core/app-mgr,
core/app-framework, and test-tools.
And rename folder component_test to component-test, update
zephyr build document.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
