English

Fixed Bugs

These are bugs in the official Emojicode compiler (master, commit 4dec45b) that this IDE fixes with a patch, so they work correctly here. If you run the same code with the official compiler, the result may be different.

Bugs that are not fixed are listed in Emojicode Language Bugs. You can see the whole patch in emojicode-wasm32.patch.

🔧 breaks Japanese text or emoji at the end of a string

In the official compiler, 🔧 (remove surrounding whitespace) breaks the last character of a string when that character is Japanese or an emoji. This happens in every environment.

The example below keeps its Japanese text on purpose, because multi-byte characters are what trigger the bug.

Emojicode
🏁 🍇
  😀 🔧 🔤  まわりの空白  🔤❗️❗️
🍉
Result in this IDE
まわりの空白
Output of the official compiler
まわりの空�

In the official compiler, the last character is broken and you get まわりの空�. The position of the last character is counted as “the first byte of that character,” so everything from the second byte onward is cut off. The patch keeps everything up to the end of the last character.

Patch: s/String.cpp
s/String.cpp
@@ -140,32 +147,41 @@ extern "C" s::String* sStringTrim(String *string) {
     for (; begin < string->count;) {
         utf8proc_int32_t codepoint;
         auto state = utf8proc_iterate(reinterpret_cast<utf8proc_uint8_t *>(string->characters.get()) + begin,
-                                      string->count, &codepoint);
+                                      string->count - begin, &codepoint);
         if (state < 0) break;
         if (utf8proc_get_property(codepoint)->bidi_class != UTF8PROC_BIDI_CLASS_WS) break;
         begin += state;
     }
 
-    size_t end = begin - 1;
+    // end is exclusive: the byte after the last non-whitespace code point. Using the start of that code point
+    // (as before) cut multi-byte characters at the end of the string down to their first byte.
+    size_t end = begin;
     for (size_t i = begin; i < string->count;) {
         utf8proc_int32_t codepoint;
         auto state = utf8proc_iterate(reinterpret_cast<utf8proc_uint8_t *>(string->characters.get()) + i,
-                                      string->count, &codepoint);
+                                      string->count - i, &codepoint);
         if (state < 0) break;
         if (utf8proc_get_property(codepoint)->bidi_class != UTF8PROC_BIDI_CLASS_WS) {
-            end = i;
+            end = i + state;
         }
         i += state;
     }
 
     auto newString = String::init();
-    newString->count = end - begin + 1;
+    newString->count = end - begin;
     newString->characters = runtime::allocate<char>(string->count);
     std::memcpy(newString->characters.get(), string->characters.get() + begin, newString->count);
     return newString;
 }
 
+#ifdef __wasi__
+extern "C" void sStringGraphemes(String *string, void *cbFn, void *cbCapture) {
+    runtime::Callable<void, s::String*> cb;
+    void *parts[2] = {cbFn, cbCapture};
+    std::memcpy(&cb, parts, sizeof(parts));
+#else
 extern "C" void sStringGraphemes(String *string, runtime::Callable<void, s::String*> cb) {
+#endif
     auto bytes = reinterpret_cast<utf8proc_uint8_t *>(string->characters.get());
     utf8proc_int32_t state = 0;
     utf8proc_int32_t prev;

Expressions that compare with 🤷‍♀️ can crash the compiler

In the official compiler, when you compile an expression containing value 🙌 🤷‍♀️ (a comparison with no value), the compiler tries to create a value it never uses, which causes undefined behavior inside the compiler. On an ordinary computer (x86-64) this happens not to cause a problem, but on WebAssembly the compiler crashes.

Emojicode
🏁 🍇
  🖍🆕 thing ⚪️
  🔤abc🔤 ➡️ 🖍thing
  ↪️ 🔲 thing 🔢 🙌 🤷‍♀️ 🍇
    😀 🔤Not a number🔤❗️
  🍉
🍉
Result in this IDE
Not a number
Output of the official compiler
(compilation never finishes)

This compares 🔲 thing 🔢 (the result of casting ⚪️ to 🔢; no value if the cast fails) with 🤷‍♀️. If you build the official compiler for WebAssembly without the patch, compiling this example never finishes (the official compiler running on an ordinary computer may happen not to show a problem).

The patch does not create the value on the 🤷‍♀️ side (it is not needed in the first place).

Patch: Compiler/AST/ASTBinaryOperator_CG.cpp
Compiler/AST/ASTBinaryOperator_CG.cpp
@@ -17,6 +17,21 @@ Value* ASTBinaryOperator::generate(FunctionCodeGenerator *fg) const {
         return generateLogical(fg);
     }
 
+    // The 🤷‍♀️ operand is never complied (its type stays noReturn), so it must not be generated:
+    // building an optional of void is undefined behaviour inside LLVM and crashes on wasm32.
+    if (builtIn_ == BuiltInType::IsNoValueLeft) {
+        auto left = left_->generate(fg);
+        return left_->expressionType().storageType() == StorageType::Box
+                ? fg->buildHasNoValueBox(left)
+                : fg->buildOptionalHasNoValue(left, left_->expressionType());
+    }
+    if (builtIn_ == BuiltInType::IsNoValueRight) {
+        auto right = right_->generate(fg);
+        return right_->expressionType().storageType() == StorageType::Box
+                ? fg->buildHasNoValueBox(right)
+                : fg->buildOptionalHasNoValue(right, right_->expressionType());
+    }
+
     if (builtIn_ != BuiltInType::None) {
         auto left = left_->generate(fg);
         auto right = right_->generate(fg);
@@ -71,14 +86,6 @@ Value* ASTBinaryOperator::generate(FunctionCodeGenerator *fg) const {
                 return fg->builder().CreateAnd(left, right);
             case BuiltInType::Equal:
                 return fg->builder().CreateICmpEQ(left, right);
-            case BuiltInType::IsNoValueLeft:
-                return left_->expressionType().storageType() == StorageType::Box
-                        ? fg->buildHasNoValueBox(left)
-                        : fg->buildOptionalHasNoValue(left, left_->expressionType());
-            case BuiltInType::IsNoValueRight:
-                return right_->expressionType().storageType() == StorageType::Box
-                        ? fg->buildHasNoValueBox(right)
-                        : fg->buildOptionalHasNoValue(right, right_->expressionType());
             default:
                 break;
         }

With 📦 files, a failed write to a file does not raise an error

In the official compiler, ✏️ (write to a file) in the files package is declared as a method that may throw an error, but its implementation (in C++) has no parameter for receiving the error. Because of this, a failed write does not raise an error. On WebAssembly, a mismatched number of arguments stops the program right there, so writing does not work at all. The patch adds the parameter and throws an error when the write fails.

Patch: files/File.cpp
files/File.cpp
@@ -37,8 +37,10 @@ extern "C" File* filesFileNewReading(String *path, runtime::Raiser *raiser) {
     return file;
 }
 
-extern "C" void filesFileWrite(File *file, Data *data) {
+// files.🍇 declares ✏️ as raising, so the compiler passes a Raiser (a missing parameter is a signature mismatch on wasm).
+extern "C" void filesFileWrite(File *file, Data *data, runtime::Raiser *raiser) {
     file->file_.write(reinterpret_cast<char *>(data->data.get()), data->count);
+    EJC_COND_RAISE_IO_VOID(!file->file_.fail(), raiser);
 }
 
 extern "C" void filesFileClose(File *file) {

The compiler and runtime disagree on the layout of class information

In the official compiler, the layout of the first part of the “class information” that each class instance holds differs between the compiler (which creates it) and the runtime (which reads it). On 64-bit environments the sizes happen to be the same, so it causes no problem, but on 32-bit WebAssembly the layout gets out of sync, and the program crashes when an instance is deinitialized. The patch changes the runtime's layout to match the compiler's.

Patch: runtime/Runtime.h
runtime/Runtime.h
@@ -48,8 +48,16 @@ using is_complete = decltype(is_complete_impl(std::declval<T*>()));
 
 }  // namespace util
 
+/// Mirrors the compiler's %runTimeTypeInfo = { i16, i16, i8 } so that the layout
+/// matches on every target (a plain pointer only happened to match on 64-bit).
+struct ClassRunTimeTypeInfo {
+    uint16_t genericParameterCount;
+    uint16_t genericParameterOffset;
+    uint8_t flags;
+};
+
 struct ClassInfo {
-    void *rtti;
+    ClassRunTimeTypeInfo rtti;
     void **dispatchTable;
     void *protocolTable;
     ClassInfo *superclass;

Stops when deinitializing a variable of a 🍱 type

In the official compiler, if you put a class instance into a variable of a type that combines several protocols (🍱Protocol Protocol🍱, see Type Values and Casting), then when that variable is cleaned up, the program calls an unrelated location as if it were a function.

Emojicode
🐊 🔊 🍇
  ❗️ 🗣 ➡️ 🔡
🍉

🐊 🏃 🍇
  ❗️ 🦵 ➡️ 🔡
🍉

🐇 🐕 🍇
  🐊 🔊
  🐊 🏃
  🆕 🍇🍉
  ❗️ 🗣 ➡️ 🔡 🍇
    ↩️ 🔤Woof🔤
  🍉
  ❗️ 🦵 ➡️ 🔡 🍇
    ↩️ 🔤Tap tap tap🔤
  🍉
🍉

🏁 🍇
  🖍🆕 pet 🍱🔊🏃🍱
  🆕🐕❗️ ➡️ 🖍pet
  😀 🗣 pet❗️❗️
  😀 🦵 pet❗️❗️
🍉
Result in this IDE
Woof
Tap tap tap
Output of the official compiler
Woof
Tap tap tap
(then stops with null function or function signature mismatch)

The start of the box holding a 🍱 value contains not “information about the box” but “a list of lookup tables, one per protocol.” Only when cleaning up does the official compiler read it as “information about the box,” so it reads past the end of the list and calls whatever value is there as a function (the result above is for a build for WebAssembly without the patch). The patch takes the cleanup function from the first lookup table in the list.

Patch: Compiler/Generation/FunctionCodeGenerator.cpp
Compiler/Generation/FunctionCodeGenerator.cpp
@@ -469,6 +469,15 @@ void FunctionCodeGenerator::manageBox(bool retain, llvm::Value *boxInfo, llvm::V
         auto conf = builder().CreateBitCast(boxInfo, typeHelper().protocolConformance()->getPointerTo());
         fnPtr = builder().CreateConstInBoundsGEP2_32(typeHelper().protocolConformance(), conf, 0, retain ? 3 : 4);
     }
+    else if (type.boxedFor().type() == TypeType::MultiProtocol) {
+        // A multiprotocol box does not store a box info but a pointer to an array of protocol conformances (one per
+        // protocol). They all belong to the boxed value's type, so the first one provides the right retain/release.
+        // Treating the array as a box info read past its end and called an arbitrary pointer.
+        auto confPtrType = typeHelper().protocolConformance()->getPointerTo();
+        auto array = builder().CreateBitCast(boxInfo, confPtrType->getPointerTo());
+        auto conf = builder().CreateLoad(confPtrType, array);
+        fnPtr = builder().CreateConstInBoundsGEP2_32(typeHelper().protocolConformance(), conf, 0, retain ? 3 : 4);
+    }
     else {
         fnPtr = builder().CreateConstInBoundsGEP2_32(typeHelper().boxInfo(), boxInfo, 0, retain ? 1 : 2);
     }

Changes for running on WebAssembly

These are not bugs, but places where this IDE changes the official compiler so that it can run inside the browser.

Adapting to a 32-bit environment

The official compiler passes a 64-bit size to malloc, which allocates memory. On 32-bit WebAssembly the types do not match, so a small function is placed in between.

Patch: Compiler/Generation/RunTimeHelper.cpp
Compiler/Generation/RunTimeHelper.cpp
@@ -111,7 +111,7 @@ void RunTimeHelper::declareRunTime() {
     buildRetainRelease(Type(Type::noReturn(), {}, Type::noReturn()), "callable.boxRetain", "callable.boxRelease",
                        boxInfoCallables_);
 
-    malloc_ = declareRunTimeFunction("malloc", llvm::Type::getInt8PtrTy(generator_->context()),
+    malloc_ = declareRunTimeFunction("ejcMalloc", llvm::Type::getInt8PtrTy(generator_->context()),
                                     llvm::Type::getInt64Ty(generator_->context()));
     malloc_->removeFnAttr(llvm::Attribute::NoRecurse);
     malloc_->addAttribute(llvm::AttributeList::ReturnIndex, llvm::Attribute::NonNull);
Patch: runtime/main.cpp
runtime/main.cpp
@@ -26,6 +26,10 @@ runtime::internal::ControlBlock* runtime::internal::newControlBlock() {
 
 extern "C" runtime::Integer fn_1f3c1();
 
+extern "C" int8_t* ejcMalloc(runtime::Integer size) {
+    return static_cast<int8_t*>(malloc(static_cast<size_t>(size)));
+}
+
 extern "C" int8_t* ejcAlloc(runtime::Integer size) {
     auto ptr = malloc(size);
     *static_cast<runtime::internal::ControlBlock**>(ptr) = new runtime::internal::ControlBlock;

The official compiler also relies on x86-64 conventions for the way it passes closures to C++. Only for WebAssembly, the way they are received is changed.

Patch: s/String.cpp
s/String.cpp
@@ -123,7 +123,14 @@ extern "C" runtime::SimpleOptional<runtime::Integer> sStringFindFromIndex(String
     return runtime::NoValue;
 }
 
+#ifdef __wasi__
+extern "C" void sStringCodepoints(String *string, void *cbFn, void *cbCapture) {
+    runtime::Callable<void, runtime::Integer, runtime::Integer> cb;
+    void *parts[2] = {cbFn, cbCapture};
+    std::memcpy(&cb, parts, sizeof(parts));
+#else
 extern "C" void sStringCodepoints(String *string, runtime::Callable<void, runtime::Integer, runtime::Integer> cb) {
+#endif
     for (size_t off = 0; off < string->count;) {
         utf8proc_int32_t codepoint;
         auto state = utf8proc_iterate(reinterpret_cast<utf8proc_uint8_t *>(string->characters.get()) + off,

Not starting processes

The compiler starts external commands for linking, but that is not possible inside the browser. Linking is done by a separately prepared WebAssembly linker, and 💻 (running a command) is made to do nothing.

Patch: Compiler/Compiler.cpp
Compiler/Compiler.cpp
@@ -93,7 +93,9 @@ void Compiler::LinkPhase::perform(Compiler *compiler) {
     auto runtimeLib = compiler->findBinaryPathPackage(compiler->searchPackage("runtime", SourcePosition()), "runtime");
     cmd << " " << runtimeLib << " -o " << outPath_;
 
+#ifndef __wasi__
     system(cmd.str().c_str());
+#endif
 }
 
 void Compiler::ArchivePhase::perform(Compiler *compiler) {
@@ -102,7 +104,9 @@ void Compiler::ArchivePhase::perform(Compiler *compiler) {
     cmd.append(outPath_);
     cmd.append(" ");
     cmd.append(objectFilePath_);
+#ifndef __wasi__
     system(cmd.c_str());
+#endif
 }
 
 std::string Compiler::searchPackage(const std::string &name, const SourcePosition &p) {
Patch: s/System.cpp
s/System.cpp
@@ -36,5 +36,7 @@ extern "C" runtime::SimpleOptional<s::String*> sSystemArg(runtime::ClassInfo*, r
 }
 
 extern "C" void sSystemSystem(runtime::ClassInfo*, s::String *string) {
+#ifndef __wasi__
     std::system(string->stdString().c_str());
+#endif
 }

No threads

Threads are not used inside the browser, so 🧵 just runs the closure you pass to it immediately (Thread_wasi.cpp).

Building the compiler itself for WebAssembly

The Emojicode compiler uses LLVM 8. The changes for building LLVM 8 for WebAssembly are in llvm-8.0.1-wasi.patch.