Gonsalves Tad

Large Language Models (LLMs) are gaining popularity and improving rapidly. Tokenizers are crucial components of natural language processing, especially for LLMs. Tokenizers break down input text into tokens that models can easily process while ensuring the text is accurately represented, capturing its meaning and structure. Effective tokenizers enhance the capabilities of LLMs by improving a model's understanding of context and semantics, ultimately leading to better performance in various downstream tasks, such as translation, classification, sentiment analysis, and text generation. Most pre-trained tokenizers are suitable for high-resource languages like English but perform poorly for low-resource languages. Dzongkha, Bhutan's national language spoken by around seven hundred thousand people, is a low-resource language, and its linguistic complexity poses unique NLP challenges. Despite some progress, significant research in Dzongkha NLP is lacking, particularly in tokenization. This study evaluates the training and performance of three common tokenization algorithms in comparison to other popular methods. Specifically, Byte-Pair Encoding (BPE), WordPiece, and SentencePiece (Unigram) were evaluated for their suitability for Dzongkha. Performance was assessed using metrics like Subword Fertility, Proportion of Continued Words, Normalized Sequence Length, and execution time. The results show that while all three algorithms demonstrate potential, SentencePiece is the most effective for Dzongkha tokenization, paving the way for further NLP advancements. This underscores the need for tailored approaches for low-resource languages and ongoing research. In this study, we presented three tokenization algorithms for Dzongkha, paving the way for building Dzongkha Large Language Models.

Unpaired image-to-image translation using Generative Adversarial Networks (GAN) is successful in converting images among multiple domains. Moreover, recent studies have shown a way to diversify the outputs of the generator. However, since there are no restrictions on how the generator diversifies the results, it is likely to translate some unexpected features. In this paper, we propose Style-Restricted GAN (SRGAN) to demonstrate the importance of controlling the encoded features used in style diversifying process. More specifically, instead of KL divergence loss, we adopt three new losses to restrict the distribution of the encoded features: batch KL divergence loss, correlation loss, and histogram imitation loss. Further, the encoder is pre-trained with classification tasks before being used in translation process. The study reports quantitative as well as qualitative results with Precision, Recall, Density, and Coverage. The proposed three losses lead to the enhancement of the level of diversity compared to the conventional KL loss. In particular, SRGAN is found to be successful in translating with higher diversity and without changing the class-unrelated features in the CelebA face dataset. To conclude, the importance of the encoded features being well-regulated was proven with two experiments. Our implementation is available at https://github.com/shinshoji01/Style-Restricted_GAN.

Developing deep learning models has a great potential in assisting human tasks involving design and creativity. This study deals with generating handwritten characters using deep learning techniques. The task is not simply generating images randomly, but generating them conditionally, making a distinction according to the UI designates. To solve this task, we constructed the Conditional DCGAN model which includes the techniques from DCGAN and Conditional GAN. We tried training the models to be able to generate conditional images by adding label information as input to the Generator. Deep learning experiments were performed using 141319 training data consisting of 96 kinds of characters including digits, Roman alphabets and Katakana. The Generator trained by inputting random noise concatenated with the 96 kinds of characters, could generate each kind of character by just adding the appropriate label information.

In this paper we propose a composite-server model and make use of the knowledge of the intrinsic composition of its service providing units (personnel or equipment) to derive Qualitative knowledge-based rules for its performance evaluation. The composite server model that takes into account the composite nature of service has wider scope in its applications and can be used to represent a variety of system classes. We use this novel concept in the performance design and improvement of collaborative engineering systems. System modeling is done by Multi-Context Map (MCM) technique. MCM is a descriptive model that expresses the collaborative activity performed through the exchange of token, material and information; bottlenecks primarily arise due to the non-uniformity in the flow of token, material and information. Another source of bottlenecks in collaborative engineering systems is the lack or surplus of service-providing units, known as "Perspectives" in the MCM terminology. Bottlenecks due to inappropriate Perspective allocation are resolved by the Qualitative Reasoning approach. We have found this method successful in the performance design, evaluation and improvement of a practical collaborative engineering system presented at the end of this paper.