ゴンサルベス タッド

Large Language Models (LLMs) are gaining popularity and improving rapidly. Tokenizers are crucial components of natural language processing, especially for LLMs. Tokenizers break down input text into tokens that models can easily process while ensuring the text is accurately represented, capturing its meaning and structure. Effective tokenizers enhance the capabilities of LLMs by improving a model's understanding of context and semantics, ultimately leading to better performance in various downstream tasks, such as translation, classification, sentiment analysis, and text generation. Most pre-trained tokenizers are suitable for high-resource languages like English but perform poorly for low-resource languages. Dzongkha, Bhutan's national language spoken by around seven hundred thousand people, is a low-resource language, and its linguistic complexity poses unique NLP challenges. Despite some progress, significant research in Dzongkha NLP is lacking, particularly in tokenization. This study evaluates the training and performance of three common tokenization algorithms in comparison to other popular methods. Specifically, Byte-Pair Encoding (BPE), WordPiece, and SentencePiece (Unigram) were evaluated for their suitability for Dzongkha. Performance was assessed using metrics like Subword Fertility, Proportion of Continued Words, Normalized Sequence Length, and execution time. The results show that while all three algorithms demonstrate potential, SentencePiece is the most effective for Dzongkha tokenization, paving the way for further NLP advancements. This underscores the need for tailored approaches for low-resource languages and ongoing research. In this study, we presented three tokenization algorithms for Dzongkha, paving the way for building Dzongkha Large Language Models.

Unpaired image-to-image translation using Generative Adversarial Networks (GAN) is successful in converting images among multiple domains. Moreover, recent studies have shown a way to diversify the outputs of the generator. However, since there are no restrictions on how the generator diversifies the results, it is likely to translate some unexpected features. In this paper, we propose Style-Restricted GAN (SRGAN) to demonstrate the importance of controlling the encoded features used in style diversifying process. More specifically, instead of KL divergence loss, we adopt three new losses to restrict the distribution of the encoded features: batch KL divergence loss, correlation loss, and histogram imitation loss. Further, the encoder is pre-trained with classification tasks before being used in translation process. The study reports quantitative as well as qualitative results with Precision, Recall, Density, and Coverage. The proposed three losses lead to the enhancement of the level of diversity compared to the conventional KL loss. In particular, SRGAN is found to be successful in translating with higher diversity and without changing the class-unrelated features in the CelebA face dataset. To conclude, the importance of the encoded features being well-regulated was proven with two experiments. Our implementation is available at https://github.com/shinshoji01/Style-Restricted_GAN.

新たな手法や技術により発展している深層学習には、人々の仕事効率を助けたり、とある現象をみて予測を立てたり、デザイナーとなって画像を生成するなど様々な可能性に満ちている。本研究では深層学習を用いて"条件付き"の手書き文字を生成を行った。単なる画像生成ではなく、入力者側が生成したい手書き文字を指定する(生成の条件をつける)ことでその文字を生成することを目的とする。目的達成のために、実験では深層学習モデルとしてDCGANとConditional GANを組み合わせたConditional DCGANを構築、ラベル情報の付加で生成条件がつけられた画像生成のトレーニングを行った。141,319のサンプル訓練データにある数字やアルファベット、カタカナなどの総計96種類の手書き文字の書き分けトレーニングを通じて、インプットの要素として含まれるランダムノイズの次元数がその種類数を上回るようにした学習済みのGeneratorは各種類に対応付けたラベルを指定するだけで、その該当文字を生成できたことを紹介する。

In this paper we propose a composite-server model and make use of the knowledge of the intrinsic composition of its service providing units (personnel or equipment) to derive Qualitative knowledge-based rules for its performance evaluation. The composite server model that takes into account the composite nature of service has wider scope in its applications and can be used to represent a variety of system classes. We use this novel concept in the performance design and improvement of collaborative engineering systems. System modeling is done by Multi-Context Map (MCM) technique. MCM is a de...