APPS FOR DEVELOPING PRONUNCIATION IN ENGLISH AS AN L2

: The goal of pronunciation teaching should be to enable learners to develop intelligible pronunciation and, in order to do this, it is important to teach perception and production of the most relevant segmental and suprasegmental features of pronunciation, considering specific groups of learners (CELCE-MURCIA et al. , 2010). Technology has played an important role in pronunciation teaching, and the applications developed for pronunciation instruction enable learners not only to engage in pronunciation activities, but to have access to a greater variety of input and immediate feedback. Having this in mind, this study aimed at analyzing the content, the pronunciation teaching steps, the features, and usability resources of pronunciation apps. In order to guide the analysis, a framework was developed based on literature related to the areas of pronunciation teaching and of Mobile Assisted Language Learning (MALL). The results showed that there is a ten - dency for the apps analyzed to focus more on segmentals. All of them offer description and analysis, listening discrimination, and controlled practice of the pronunciation features, as well as feedback. However, they were limited in terms of guided and communicative practice, of Automated Speech Recognition (ASR), and of variety of input.


INTRODUCTION
Pronunciation is one of the core skills of speaking which is necessary for successful communication to take place, and the lack of instruction might result in lack of confidence to speak, or difficulties to understand and be understood in the Second Language (L2).
A realistic goal in pronunciation teaching is to enable learners to "surpass the threshold level so that pronunciation will not detract from their ability to communicate" (CELCE-MURCIA et al., 2010, p. 9), and also to make their communication more intelligible.
In order to reach that goal, learners must be provided with opportunities to practice perception and production of the most important segmental and suprasegmental features of pronunciation (CELCE-MURCIA et al., 2010;KELLY, 2001) through activities that include presentation, listening, and practice focused on both form and meaning, all of them followed by feedback (CELCE-MURCIA et al., 2010).
Since the 1960's, studies have investigated how technology can be applied in order to enhance language learning (BAX, 2003;CHAPELLE;JAMIESON, 2008;GRUBA, 2006;LEVY;HUBBARD, 2005). As technology, once restricted to computers, has not only evolved but gone mobile, the field of Mobile Assisted Language Learning (MALL) emerged. In MALL, mobiles are devices which are portable and personal, and UNIVERSIDADE FEDERAL DO PARANÁ Departamento de Letras Estrangeiras Modernas ISSN: 1980-0614 these features added to their connectivity enable learners to practice the L2 at the most suitable time and place for them. This may not only increase the time engaged in language learning activities, but also allow learning to happen in more naturalistic settings, which could lower the barriers between what happens in the classroom and in students' lives . The applications, or apps, which focus on language learning seem to be helpful to support pronunciation instruction, providing practice of receptive and productive skills of pronunciation and offering immediate feedback, in an environment which allows for comfort and unlimited attempts toward confidence (GUO, 2014).
Regarding the development of L2 pronunciation through mobile devices, studies (ARAGÃO; PAIVA; JUNIOR, 2017; GONZALEZ, 2012;GUO, 2014;PAIVA 2017PAIVA , 2018SALBEGO;SARAN;SEFEROGLU;CAGILTAY, 2009;SUN et al., 2017) have shown that mobile phones and general language learning apps have the potential to develop L2 pronunciation. However, as the apps proliferate, it becomes essential to understand how they differ from one another, what their features are, and what pedagogical benefits may be derived from their use (KUKULSKA-HULME; LEE; NORRIS, 2017). Thus, this study aims at that understanding by investigating four apps -English Pronunciation Tutor, EnglishPronunciation, Elsa, and Juna -available for developing pronunciation in English as an L2.
In so doing, this study contributes to the area of pronunciation teaching, which is still considered overlooked by language teaching materials and classroom practice (ALBINI; KLUGE, 2011;SILVEIRA, 2004;STANLEY, 2013).

Pronunciation instruction
Pronunciation has had different roles throughout the language teaching methods (CELCE-MURCIA et al., 2010;KERMAD, 2018;SILVEIRA, 2004), and today it is generally agreed that the goal of pronunciation teaching should aim at intelligibility (ALVES, 2015;CELCE-MURCIA et al., 2010;SILVEIRA et al., 2017), given that aiming at native-like pronunciation is incongruent with empirical evidence (SILVEIRA et al., 2017).
In order to reach that goal, learners must have opportunities to practice perception and production of the most relevant aspects of segmental and suprasegmental features, once the inaccurate use of any of them has the potential to inhibit successful communication. By relevant aspects, Celce-Murcia et al. (2010) explain that not all aspects of L2 pronunciation should be approached to every group of learners, being the teacher responsible for deciding what is pedagogically meaningful for the given group. ISSN: 1980-0614 Regarding connected speech in English, for instance, C to V linking, V to V linking, consonant assimilation, and palatalization should be all highlighted, as they frequently occur in spoken English language. Also, concerning word stress, only three levels of word stress should be taught instead of six, as not all levels are discernible and, therefore, are not useful for pedagogical purposes (CELCE-MURCIA et al., 2010). Celce-Murcia et al. (2010) claimed that despite the Communicative Approach having stated intelligible pronunciation to be the goal of pronunciation teaching, no set of strategies or methodology for doing it was included. The authors, then, proposed a framework for teaching pronunciation, which is grounded on the principles of the Communicative Approach. The framework recommends a division of the pronunciation lesson into five steps which start by providing learners with analytical information and awareness raising of the pronunciation features, followed by listening discrimination, and three different types of practice: controlled, guided, and communicative.

UNIVERSIDADE FEDERAL DO PARANÁ
More specifically, the steps are: 1) description and analysis -oral and written illustrations of how the pronunciation feature is produced and when it occurs within spoken discourse; 2) listening discrimination -focused listening practice with feedback on learners' ability to correctly discriminate the pronunciation feature; 3) controlled practice -oral reading of minimal-pair sentences, short dialogues, etc., with special attention paid to the highlighted pronunciation feature in order to raise learner's consciousness; 4) guided practice -structured communication exercises, such as information-gap activities or cued dialogues, which enable the learner to monitor for the specified pronunciation feature; and 5) communicative practice -less structured, fluency-building activities (e.g., role play, problem solving) that require the learner to attend to both form and content of utterances.
The authors emphasized that each step plays a key role in the acquisition of new pronunciation features. As pronunciation learning is a complex and nonlinear process (LIMA JR; ALVES, 2019), the complete framework is meant to be applied throughout several lessons and every step can be revisited whenever necessary. In addition, they asserted the importance of systematic feedback in all stages. According to Celce-Murcia et al. (2010), feedback for description and analysis is provided on the placement of articulatory organs. Regarding listening discrimination, learners are made aware if they have correctly identified the target sound. Once the goal of controlled practice is accuracy, feedback may occur at any time and it may be delivered by the teacher or peers. During guided and communicative practice, as the goal of the activity is communication, feedback tends to be delayed until the end and may also be delivered by the teacher or other learners. ISSN: 1980-0614 Within the framework, the presentation of phonetic transcription for pronunciation instruction is also considered depending on the group of learners, as it may allow them to comprehend the elements of pronunciation visually and aurally, and to promote their learning autonomy (CELCE-MURCIA et al., 2010;MARTINS, 2015). In order to investigate whether resources for pronunciation instruction follow the pronunciation teaching steps proposed by Celce-Murcia et al. (2010), it is necessary to examine whether description and analysis; listening discrimination; and controlled, guided, and communicative practice are enabled by these materials, as well as whether feedback is provided in every step.

UNIVERSIDADE FEDERAL DO PARANÁ
Once the goals of pronunciation instruction and a framework for its development have been briefly presented, the potential of apps and their features to enhance pronunciation development are presented next.

MALL and L2 pronunciation
Technology permeates many aspects of our lives, including language learning resources to be used to access information, get exposure to a target language, seek entertainment, communicate and interact, manage learning, and contribute for learners to feel more motivated and engaged (STANLEY, 2013). As technology has gone mobile, Mobile Learning (ML) has become a reality. Kukulska-Hulme and Shield (2008, p. 3) define ML as "learning mediated via handheld devices and available anytime, anywhere", being both formal and informal.  claimed ML occurs predominantly out of class environments, but acknowledged it may happen in both contexts. Likewise, Kukulska-Hulme, Lee, and Norris (2017, p. 217) affirmed that "although mobile learning offers certain benefits in the classroom, the use of mobile devices also potentially extends learning beyond the classroom setting".

Mobile Assisted Language Learning (MALL) is the area of research concerned
with ML practices that focus on language learning. Among the possibilities for the use of mobile devices for language learning, several apps have been developed, focusing on general or specific language skills/aspects. Regarding the ones developed for pronunciation, they may present a set of MALL features proposed by Stockwell and Hubbard (2013), and the ones relevant for the development of L2 pronunciation are presented next.
Pronunciation apps may present many features designed to assist the development of pronunciation. One is the exposure to a variety of input on demand (LEVIS, 2007), as learners may have access to different varieties of English, regional accents, and male/female UNIVERSIDADE FEDERAL DO PARANÁ Departamento de Letras Estrangeiras Modernas ISSN: 1980-0614 voices at their fingertips, at the most suitable time and place for them. In this sense, they allow the use of many pronunciation models which are needed to increase communicative flexibility and respect for accent diversity (CELCE-MURCIA et al., 2010;LEVIS, 2007).
The apps may have a feature that allows the selection of users' L1, making them customized to the learner and compensating the L1 effect on the development of L2 pronunciation. This way, exercises and materials most relevant for the learner with that specific L1 may be provided, taking into consideration possible cross-linguistic influences which may hinder intelligible pronunciation, for instance.
In addition, the apps may include a feature that provides a proficiency test in order to identify users' main difficulties, or simply to allow for the selection of the level of difficulty and aspects of L2 pronunciation for practicing. This contributes as a prioritysetting feature. According to Munro and Derwing (2015a p. 393), "the common one-sizefits-all approach in which practice is offered in 'everything' is unhelpful to teachers and students who need to focus their attention on issues that will genuinely improve their communication skills". Therefore, the possibility of selecting user's L1 and also different levels of lessons when using the apps are important aspects to be taken into account regarding pronunciation apps.
Moreover, the multimodal environment of apps may allow for the presentation of the selected pronunciation features in a variety of ways, for instance, through the use of textual information, illustrations, learner-friendly diagrams, and videos. The media must be well designed, otherwise the apps may be ineffective. For Pires and Tumolo (2020), apps may sometimes display pictures which are not relevant to the activity proposed, confusing the learners. Similarly, Kukulska-Hulme, Lee, and Norris (2017) have revealed the incongruity of meaning between the modes of language and visuals in a commercial vocabulary app, with one-fifth of the images being unclear, decontextualized, and potentially confusing for users. Considering a minimal-pair pronunciation activity, for instance, a picture which does not easily relate to the given word may be a problem for the learner. Also, not only is the choice of pictures important, but also their quality, given that a blurry picture or in an inadequate format may also affect understanding (CHINNERY, 2006). All this points out to the need for including images relevant to the activity, congruence between language and visuals, pictures easily relatable to words, and clear and adequate pictures. With this embedded feature, pronunciation apps may better assist learners' to develop their pronunciation.
Another important feature of the apps is the voice. According to Mayer (2009, p. 256), "a machine-synthesized voice -although perceptually discernable -may not UNIVERSIDADE FEDERAL DO PARANÁ Departamento de Letras Estrangeiras Modernas ISSN: 1980-0614 convey as much sense of social presence". In the same way, Hinks (2015) affirmed that the greatest research challenge at present is to improve the naturalness of the sound and pronunciation. As the voice present in pronunciation apps can often sound quite artificial, developers have been wary of using it as a teaching model, preferring recordings of natural voices. Therefore, the choice and quality of videos, illustrations, pictures, and voices must be included in any analysis of pedagogical resources developed for pronunciation instruction.
The Automated Speech Recognition (ASR) can be considered another relevant feature of pronunciation apps; with it, learners may practice pronunciation and receive immediate feedback, that is, feedback just in time for learning. There may be different types of feedback provided by the apps, from "right/wrong", to "amount of % correct" or "% amount of native likeness", sounds such as clapping hands, or even visual feedback for showing an approximation of intonation contour, for instance. As previously discussed, feedback is required in all steps of the framework for teaching pronunciation (CELCE-MURCIA et al., 2010), and, according to Gonzalez (2012, p. 86), "app users should always know why they have made the mistake and, if possible, be given suggestions for improvement". KUKULSKA-HULME; SHIELD, 2008).
In addition to the features previously discussed, which result from research in the field of MALL, an analysis of pronunciation apps must also be concerned with their usability, that is, the ease of using them. Apps must be clear and self-explanatory, developed in a way so that the user is able to use it without effort and doubts. Krug (2008) affirmed that every doubt during use may distract the learner from the target task. The author also mentioned the importance of having a balanced amount of information on the screen, which must also be well hierarchized, so the user is not overwhelmed and is able to guide him/herself during the use. For this reason, the usability of pronunciation apps must also be taken into consideration in any analysis of these pedagogical materials.
In sum, a broad analysis of pronunciation apps must look into its content, that is, whether it includes the most relevant segmental and suprasegmental features of pronunciation, and also into the pronunciation teaching steps adopted by it. In addition, it must be concerned with the features and usability resources incorporated by them in order to promote pronunciation development. Investigating which of these aspects are included in the apps English Pronunciation Tutor, EnglishPronunciation, Elsa, and Juna, is the purpose of this study.

METHOD
This section provides the method used for the study. It presents, first, the steps and criteria for the selection of the apps; second, the description of the framework developed for the analysis; and third, the description of the analysis, including the scoring procedure.

Selection of the apps
A search was carried out on App Store and Google Play, which are currently the two most popular app stores, with the following key-words: English pronunciation, learn pronunciation, and English accent. A number of 250 apps were found -this number is not related to 250 different apps, however, as some of them were available on both App Store and Google Play. Among all apps found in the search, the ones under the following categories were excluded: a) reference apps, e.g. dictionaries and translators; b) apps not designed specifically for pronunciation instruction; c) apps with only the International Phonetic Alphabet (IPA); d) apps developed for learners of a specific L1; and e) apps with problems after installed.
Four apps remained and were selected to be analyzed, namely: English Pronunciation Tutor, Elsa, EnglishPronunciation, and Juna. All of them have been developed for pronunciation instruction and are either free -that is, with access to all of its content for free -or freemium, meaning that may provide a free trial -usually a week, or a month, requiring the payment after this period, or they may offer some of its content available for free, being necessary to pay in order to have full access of the content. The apps were downloaded and installed. Considering this research proposed to analyze all features available in the apps, full access to their content was purchased when necessary.
Finally, it is important to mention that all data were collected from the apps in the period

App Store Google Play Free Free trial Free limited access
English Pronunciation Tutor Source: The authors (2021). It is divided into three categories, namely: 1) Content; 2) Pronunciation teaching steps;

Development of the framework for analysis
and 3) Features and usability. The questions in the framework are available as appendix 1, together with the results for each app analyzed.
The first category, Content, refers to the pronunciation features to be taught, that

Data analysis
Each item in the framework developed for this research received a score of 0 (when the app did not present the item) or 1 (when it presented the item). This way, a chart with the score obtained by each app in the three categories (Content, Pronunciation teaching steps, and Features and usability) is presented, providing information on the performance of every app in each of these categories. A chart is also provided for all the four apps analyzed, allowing the reader to compare all performances. As all items included in the framework developed for this analysis are considered relevant to the teaching of pronunciation in general, they were equally weighed in this research. When considering specific groups of learners, however, these aspects may be weighed differently, according to the learner's own difficulties and goals.
Notes and screenshots were taken as evidence regarding each of the items in the three categories of the framework in order to carry out the qualitative analysis. This way, it was possible to look into how the pronunciation features were included in the apps, the sources for presentation and practice for such features, if and how the apps provided feedback, as well as into the features and usability resources incorporated by them.

RESULTS
This section provides the analysis for the four pronunciation apps. Each app is analyzed considering, first, the Content, that is, the segmental and suprasegmental features focused; second, the Pronunciation teaching steps; and third, the Features and usability of the apps.

English Pronunciation Tutor
English Pronunciation Tutor (EPT), in terms of content, enables the user to practice all segments of English, individually and in contrast, as well as to attend to the differences resulting from the positional variation in some of them. The only suprasegmental feature covered by the app is the stress in words and sentences, however. For this reason, the app scored 63% under the category Content. : 1980-0614 Concerning the Pronunciation teaching steps adopted by the app, all units in the app offer description and analysis of the pronunciation features in varied ways, by using textual information, narration, and visual representations. Listening discrimination is also present in all lessons, as users are able to raise awareness of the pronunciation features presented and, in some lessons, may also discriminate sounds in the Listening Quiz section. Controlled practice of pronunciation is present in all lessons as well, through

UNIVERSIDADE FEDERAL DO PARANÁ
Practice and in some lessons through Speech Recognition, sections where users are able to record their production of words and sentences.
Having said that, it is possible to conclude that the app does not go beyond the third step of the framework for teaching pronunciation adopted in this study, as it focuses mainly on accuracy and does not provide guided or communicative practice of the pronunciation features. Even though feedback is provided by the ASR feature, it has limitations, such as not recognizing varieties which deviate from the native-like form, or ignoring external noises. Hence, the app scored 29% in the Pronunciation teaching steps category.
The app scored 57% in the third category -Features and usability. Some features incorporated by the app to promote pronunciation development are, for instance, the use of illustrations in order to explain the articulation of the organs and to illustrate words containing the targeted sounds. Some variety of input is provided by the app, and its lessons are within the time recommended for MALL materials. Concerning the usability of the app, its quantity of information per screen is balanced and well hierarchized, and the app presents clear icons and directions for the user, who should be able to navigate it without effort. Figure 1 illustrates the score obtained by EPT.

Elsa
The content available in Elsa presents all segmental and suprasegmental features expected to be included in pedagogical resources for pronunciation instruction. Therefore, the app scored 100% in this category.
Elsa provides the user with description and analysis related to the characteristics of all sounds, and the suprasegmental features of connected speech, word stress, and sentence stress. However, it lacks explanation regarding prominence and the relationship between intonation and meaning. Listening discrimination and controlled practice are available for all features of pronunciation covered by the app, but Elsa does not provide its users with opportunities for guided or communicative practice, since it focuses on accuracy. Feedback is available in Elsa and, despite some limitations, it can be effective, once it is able to ignore noises, to tell the users what the mispronunciation is and to provide them with guidance on what to do in order to improve production. For this reason, Elsa scored 58% in the category Pronunciation teaching steps.
As for the features and usability resources incorporated by the app, Elsa scored 93% in this category. The app provides some variety of input, asks for the users' L1, provides a proficiency test, and allows users to select the level of proficiency during the exercises. The presentation of pronunciation features is done through a variety of ways, such as with the illustrations, videos, and narration. In addition to this, Elsa has the push feature, and its lessons are within the recommended time for MALL materials.
These features incorporated by the app are relevant, as they contribute to prevent the so-called one-size-fits-all approach (DERWING; MUNRO; 2015b) commonly found in pronunciation instruction digital materials. Thus, the app may assist learners who want to develop their English pronunciation, as it enables them to focus on specific goals and needs they may have. Elsa's score in each category analyzed can be seen below.

Juna
Juna enables its users to work with all the sounds of English language individually, in contrast, and in different positions within words. The app focuses exclusively on segmentals, as it is described on the App Store and on its official website.
The score obtained by Juna under Content, then, was 50%, as it does not cover any of the suprasegmental features which are expected to be included in pedagogical resources for pronunciation instruction.
Description and analysis, listening discrimination, and controlled practice of the segmental features are available and, thus, Juna goes up to the third step for teaching pronunciation adopted in this study. Therefore, the app only enables practice focused on accuracy, with no opportunities to focus on meaning and exchange of information, which would be the goal of activities within guided and communicative practice (CELCE-MURCIA et al., 2010). The ASR in Juna provides feedback on user's production, and this feature is also able to ignore noises, possibly encouraging learners to use the app anywhere, without previously planning. However, limitations have been found in what concerns this feature. For instance, there are times that the app identifies and transcribes the accurate production, but provides negative feedback without indicating the cause for the mispronunciation, nor providing instructions on how to improve the production. This may lead L2 learners to confusion. For this reason, the app obtained a score of 23% in the category of Pronunciation teaching steps.
The score obtained in the Features and usability category was 71%. The app offers some variety of input to its users and is embedded with a feature that allows them to select the level of activities in certain lessons, thus, being able to focus on pronunciation aspects  The following chart presents the scores for all the four apps analyzed: Next, as part of the analysis, a discussion is provided for all the four apps analyzed. The four pronunciation apps analyzed in this study provide description and analysis, listening discrimination, guided practice, and feedback. In spite of this, they do not go beyond the third step of the framework for teaching pronunciation communicatively (CELCE-MURCIA et al., 2010), once they do not provide guided or communicative practice, focusing on meaning and exchange of information. If the apps offered this type of practice instead of practice focused on accuracy only, they would also be an efficient tool to support learners develop fluency, also considered crucial for intelligible pronunciation. This limitation may be because the apps themselves were not developed with this purpose, or due to technological constraints, which may be easily overcome with the advancement of technology. Perhaps in the near future the apps will allow learners to interact with each other or with the teacher in order to practice pronunciation communicatively, or even include artificial intelligence technology, so that the learner may interact with the app itself. In the meantime, the use of such apps could be combined with other activities that can be delivered through mobile devices and other apps, such as WhatsApp, Instagram, Facebook, Siri, Google Assistant, and Alexa, in order to engage in communicative practice. This way, the apps analyzed in this study could have their use supplemented by activities focused on expressing meaning and exchange of information, such as cued-dialogues, simple information-gap activities, strip stories, storytelling, debates, and interviews.

UNIVERSIDADE FEDERAL DO PARANÁ
In relation to the description and analysis as well as feedback provided by the apps, they are usually more detailed regarding the segmental lessons. The listening discrimination and guided practice of segmental lessons also outnumber the ones related to the suprasegmentals, when they are approached by the apps. It is possible to identify, hence, that there is a trend in the apps to focus more on segmentals, a practice which permeated throughout many language teaching methods (CELCE-MURCIA et al., 2010). Even though scholars affirm that pronunciation instruction is moving away from a segmental/suprasegmental debate towards a more balanced view nowadays (CELCE-MURCIA et al., 2010), it seems that it is not the case of the apps analyzed in this study.
Therefore, pronunciation apps might follow the conclusions of research and provide balanced practice regarding all features of pronunciation, once all of them may affect intelligible pronunciation (CELCE-MURCIA et al., 2010). Feedback was provided by all the apps since they are embedded with an ASR feature. Nevertheless, the ASR feature has limitations. For instance, none of the apps recognize varieties which deviate from the native norm, and some are not able to ignore external noises, making it difficult for someone to use anywhere, as it is proposed by MALL (CHINNERY, 2006;. Furthermore, the ASR feature sometimes identifies a correct production of the user, that can be visualized by the transcription, but provides incorrect feedback through sounds and/or images. There are also times when ASR is simply not able to identify what was said by the user. This limitation may cause the app to provide incorrect feedback to the learner. As pointed out by Levis (2007), the negative consequence of wrong feedback may confuse learners and hinder learning. This may be the case when learners receive incorrect feedback by the ASR. Once they may not be able to understand the cause of the mispronunciation or to consider the ASR may be providing incorrect feedback, they may feel frustrated about their performance and feel insecure about using the app or even speaking in the L2.
Some features of MALL adopted in this study are provided by the pronunciation apps analyzed, such as the variety of input through different narrators (LEVIS, 2007), with male and female voices. However, the lessons within the apps do not bring phonological variations beyond North American English or British English. All the apps analyzed in this study make use of images, illustrations, animations, videos, textual and aural information, being most media relevant for what it has been proposed (KUKULSKA-HULME; LEE; NORRIS, 2017). Also, most voices found in the apps sound natural, thus, conveying the idea of a social presence for the users (MAYER, 2009), which may motivate learners to use the apps for developing their pronunciation.
Bringing more phonological variations than it is usually found in traditional L2 classrooms, allowing learners to focus on their main difficulties and goals, as well as encouraging them to take at least a short pronunciation lesson every day are features that may contribute not only for developing learner's autonomy, but to increase their motivation and reduce the anxiety which may be related to speaking and pronunciation in an L2. Therefore, pronunciation apps that include some or all of these features may  Despite their limitations, the apps analyzed seem to be a helpful pedagogical resource for working with presentation, awareness raising, listening, controlled practice, and providing feedback regarding pronunciation features.